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CHAPTER  1 
INTRODUCTION 

1.1  Motivation 

Much  of  the  recent  work  in  information  theory  hzis  involved 
multiple  user  channels.  Such  configurations  seem  natural  in  view  of 
modem  communication  environments,  for  exaBq>le  satellite  or  data  net- 
works. In  addition  the  results  obtained  for  multiple  user  channels  are 
sufficiently  different  in  character  from  those  in  single-source 
single-destination  problems  to  justify  interest  in  the  area  from  a 
purely  theoretical  standpoint.  A reasonably  coop  lets  survey 
of  known  results  in  multiple  user  comDunlcaticn  problems  is  contained 
in  tl]. 

In  a multiple  user  problem  the  notion  of  the  capacity  of  a 
channel  is  extended  to  that  of  em  admissible  rate  region,  or  capacity 
region.  In  a most  general  setting,  each  of  M data  sources  wishes  to 
communicate  reliably  with  each  of  N data  sinks  over  some  given  M**input 
N-output  channel.  An  admissible  rate  vector  is  a point  in 

(the  positive  orthant  of  real  MN  space)  such  that  the  i^  transmitter 
can  ccamunlcate  to  the  receiver  with  arbitrarily  small  probability 
of  error  for  all  i,j  at  rates  R^^.  The  closure  of  all  such  is 

the  capacity  region  of  the  channel.  A further  generalisation  sometimes 
considered  is  to  allow  the  messages  to  have  seme  correlation.  That  is, 
two  or  more  transmitters  mi^t  have  one  comson  message  intended  for  a 
receiver,  i^ic^  they  can  conmnnicate  cooperatively,  in  addition  to  a set 
of  independent  messages.  The  dual  situation  is  also  possible i one  trans- 
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mitter  might  have  a coomon  message  Intended  for  all  or  some  subset  of 

the  receivers,  in  addition  to  private  messages  for  each.  * 

Multi-user  problems  are  rarely  considered  in  such  generality. 

Two  canonical  problems  of  both  theoretical  amd  practical  slgnificwce  are 

the  two  user  broadcast  channel  (BC)  in  which  one  transmitter  wishes 

to  coomunicate  separate  information  to  two  distinct  receivers,  and  the 

two  user  multiple  access  channel  (MAC)  in  which  two  transmitters  wish 

to  coomunicate  separate  information  to  one  receiver.  Admissible  rate 

2 

regions  in  these  cases  are  subsets  of  the  positive  quadrant  of  R . 

In  this  dissertation  we  extend  these  models  in  that  we  allow 
feedback  from  receivers  to  transmitters.  The  results  of  the  use  of 
feedback  are  interesting  since,  unlike  the  single-input  single-output 

a 

channel,  where  it  is  known  [32]  that  feedback  does  not  Increase  the  ca- 
pacity of  memory less  channels,  it  has  been  shown  that  in  the  case  of 
memoryless  MAC'S,  feedback  can  increase  the  capacity  region  ([12], [13]). 

On  the  other  hand,  it  hem  been  shown  that  for  at  least  one  broadcast  model, 
that  in  which  one  channel  is  a physically  degraded  version  of  the  other, 
feedback  does  not  Increase  capacity  ([8], [16]).  Although  El-Gamal  in 
[8]  conjectures  that  this  result  holds  true  for  more  general  broadcast 
channels,  it  will  be  shown  in  this  dissertation  that  this  conjecture  is 
false . 

The  results  presented  in  this  dissertation  will  involve 
deterministic  coding  schemes  for  additive  white  gaussian  noise  (ANOI) 
multiple  user  channels  with  feedback,  whose  operation  require  that  the 
feedback  be  noiseless.  The  coding  schemes  are  extensions  of  the  feed- 
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baclc  coding  scheme  for  single  user  A.WGM  channels  developed  by  Schallcwljk 
and  Kad-lath  [2]  and  Schallcwljk  [3],  which  also  required  noiseless  feed- 
back. The  noiseless  assumption  might  be  well  justified  for  exanple,  in 
the  case  of  satellite  communication,  where  the  satellite-to-ground  link, 
inherently  power  limited,  is  considerably  more  error  prone  than  the 
reverse  link,  over  which  ground  stations  can  operate  at  much  higher  power. 

The  results  obtained  here  are  quite  interesting  in  that  the  codes 
are  deterministic  and  reasonably  simple  to  Inplement,  and  yield  error 
rates  vAiich  approach  zero  with  doubly  exponential  behavior  in  block  length. 
In  addition  the  data  rates  achieved  both  disprove  the  conjecture  men- 
tioned above  for  the  BC,  and  also  exceed  previously  known  achievable 
rates  for  the  AWGN  MAC.  Indeed  for  this  MAC  we  will  show  that  the  set 
of  achieveUsle  rates  coincides  with  the  capacity  region. 

Before  proceeding  we  summarize  two  basic  multi-user  techniques 

which  will  be  useful  in  the  sequel.  The  first  is  time-sharing,  nie 

11  2 2 

time  sharing  argument  is  that  if  two  rate  pairs  (Rj^,R2) 
achievable,  any  point  of  the  form 


- orJ  + {l-a)R^ 
Pj  - asj  + (l-a)R^ 


(1.1.1) 


is  achievable  for  all  a between  0 and  I.  The  intezamdiate  point  is 

achieved  by  e]q>loying  a code  which  achieves  for  a fraction  a 

2 2 

of  the  time,  and  a code  which  achieves  (l^»^)  for  the  remaining  1-a. 
One  consequence  of  the  time-sharing  argument  is  that  ci^aclty  regions 
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I 


u 


I 


are  always  convex.  Another  is  that  in  cases  for  which  the  capacity 
region  is  a polygon,  the  achievability  of  the  extreme  points  suffices 
to  demonstrate  the  achievability  of  the  entire  capacity  region.  Another 
property  derivable  from  time  sharing  arguments  is  that  if  a point  (R^,R^) 
is  achievable,  then  all  points  which  satisfy 


0 1 ®1  1 *4. 

0 < R^  < R- 


(1.1.2) 


eure  also  achievable.  Therefore,  in  general,  achieving  points  on  the 
boundary  (in  of  the  achievable  region  suffices  to  demonstrate  the 
achievability  of  the  entire  region. 

nie  other  technique  which  is  useful  is  superposition.  The 
exact  form  of  the  superposition  argument  vaurles  with  the  specific  appli- 
cation, but  essentially  it  is  aa  follows:  when  two  Independent  codewords 
eure  combined  by  some  method  appropriate  to  the  structure  of  the  channel, 
one  codeword  may  be  decoded  by  treating  the  other  as  noise  (that  is, 
part  of  the  randomization  iiqparted  by  the  channel) . Once  this  code- 
word is  known  (at  least  with  hi^  probability)  its  effect  can  be  removed 
by  the  decoder,  and  the  other  codeword  detected  as  though  the  first  were 
known.  See  [4]  for  a discussion  of  the  superposition  argument  in  the 
broadcast  contoxt. 

In  the  remainder  of  this  chapter  we  will  discuss  the  two  models 
to  be  considered,  the  nature  of  previously  known  results,  md  the 
Schalkwljk-lCsd.lath  coding  sdieme.  The  fundamental  information  theoretic 


-13- 

quantities  used  below  are  as  in  Gallager  [24] . Non-standard  or  ambi- 
guous terminology  will  be  defined  when  necessary. 

1.2  Backgroiind 

1.2.1  "nie  Multiple  Access  Chaimel 

The  MAC  was  first  discussed  by  Ahlswede  [9]  and  Liao  [10] . A 
fairly  coii{>lete  discussion  generalized  to  include  the  presence  of 
correlated  sources  was  contributed  by  Slepian  and  Wolf  [11] . 

Figure  1.1  shows  a general  discrete  time  MAC  (with  dashed  lines 
representing  feedback  links) . nie  channel  output  variable  at  time  k, 
denoted  by  is  a random  function  of  the  k input  letters,  and 
^2k'  ^o^J^ed  by  a time-invariant  memoryless  conditional  probability 
law  pCylxj^x^).  The  capacity  region  without  feedback  is  given  by 

C - co[  U fC{p)]  (1.2.1) 

peP 

where  co[  ] denotes  closure  of  the  convex  hull,  K(p)  is  given  by 

K(p)  - {(Rj^,R2)t  £1(Xj^;Y|X2) 

Pj  < 1(X2>y1Xj^)  (1.2.2) 

and  P is  the  set  of  probability  assignments  on  (X^,X2)  for  idiich  X^  and 
X^  euw  statistically  independent.  In  the  sequel  we  will  omit  the  cus- 


tomary but  redundant  imposition  of  a lower  bound  of  zero  for  rates. 
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Analogous  expressions  hold  for  some  continuous  asplitude 
channels  with  suitable  constraints  on  the  inputs.  For  the  additive 
vdiite  gaussian  noise  channel,  pCylx^^x^)  ~ N(x^+x^,(J^) , where  p(  ) is 
in  this  case  a probeJsility  density  function,  resulting  from  the  addition 
of  an  independent  gaussian  noise  variable  with  vetriance  to  the  sum 
of  Xj^  and  x^.  Under  the  constraint  that  E[X^]  £ for  i ■ 1,2  (1.2.1) 
and  (1.2.2)  reduce  to 


1 

C - { (Rj^,R2)  ; < j ln(l  + — ) 


1 2 
<2  ln(l  + ^) 


(1.2.3) 


Figure  1.2  shows  the  capacity  region  for  Pj^/a*  - P^/O*  ■ 10.  This 
result  was  shown  in  [25]  and  [19]. 

Svqperposition  arguments  can  be  used  to  justify  the  achieva- 
billty  of  the  entire  region  in  Fig.  1.2.  Points  A and  B can  be  obtained  by 
allowing  only  one  transmitter  to  eommunieate , at  capacity.  Point  C is 
obtained  by  allowing  transmitter  1 to  transmit  with  power  using  a code 
whidt  ie  reliably  deeodabla  in  the  pxesenoe  of  noise  with  variance  P^x?*. 
nien  if  the  code  letters  (kBl,...,N  and  i«l,2)  are  sequences  of 

gaussian  random  variables  with  variance  P^,  reliable  transmission  of  the 
message  sent  by  transmitter  1 is  possible  at  rates  up  to 
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- H(y)  - h(y|Xj^) 

- J In  2iTe(Pj^+P2+o*) 

1 **1 

- i In  ( 1 + L.  ) 

2 B ^2 


- j In  2Tre(P2+a*) 


(1.2.4) 


(Xice  message  1 Is  decoded,  {xj^j^}^can  be  subtracted  from  the  received 
data.  Then  message  2 may  be  reliably  decoded  at  rates  to 


Rj  - H(y|Xj^)  - HCyIXj^X^) 

■ j In  2ire(Pj^+C*)  - i In  2irea* 

1 ^1 

- i In  ( 1 + -i  ) 

2 a* 


(1.2.5) 


This  argument  may  be  reversed  to  <^taln  point  D.  The  remainder  of  the 
capacity  region  may  be  achieved  by  the  use  of  time-sharing. 

The  addition  of  feedback,  as  mentioned  above,  can  enlarge  the 
capacity  region.  This  result  was  first  shewn  using  an  ad  hoc  scheme 
for  the  noiseless  binary  erasure  MAC  by  Gaarder  and  Wolf  [12] . This 
channel  has  Input  alphabets  * ^2  * output  alphabet  / >■  {o,l,e}, 

and  a channel  probability  function  given  by  p(o|oo)  > p(l|ll)  <■  p(e|oi)  « 
p(e|lo)  > 1.  That  Is,  when  the  Inputs  agree,  the  output  equals  their 
connon  value,  and  when  they  disagree  an  eruure  occurs.  For  this 
channel,  the  region  specified  by  (1.2.2)  Is 


*2ll 


(1.2.6) 


C 
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R^+R^  1*5  bits/transmissicai 


(1.2.6) 


Roughly,  the  approach  of  [12]  is  as  follows:  let  the  transmitters 


independently  choose  sequences  of  input  letters  of  length  N^,  where  each 


letter  is  equiprobably  0 or  1.  They  then  send  these  bits  over  the  chan- 


nel. On  the  average  N^/2  tremsmissions  will  result  in  erasures.  For 


each  ereuiure,  specifying  either  transmitter  I's  or  transmitter  2's  bit 


will  resolve  the  receiver's  uncertedjity.  By  the  use  of  feedback  both 


transmitters  know  which  bits  were  erzised,  and  what  transmitter  I's  in- 


tended bit  was  for  each  transmission  (transmitter  1 does  since  he  sent 


it  and  transmitter  2 does  since  it  disagreed  with  his  own) . The  trans- 


mitters must  now  resolve  N^/2  bits  of  uncertainty  in  transmissions. 


With  the  transmitters  sending  a common  message,  the  channel  is  noiseless 


%rith  ternary  ou^ut  and  quaternary  input.  It  may  therefore  be  used 


at  rates  to 


H(y)  ■ log^S  bits/transmission 


(1.2.7) 


so  that  bits  can  be  conveyed  in 


2 2 log^a 


transmissions . 


(1.2.8) 


Therefore  the  rate  pair  achieved  is  given  by 


R,  ■ bits/transmission 

X 2 + Nj 


(1.2.9) 


q 
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which  lies  outside  the  region  given  by  (1.2.6). 

Since  the  appeeirance  of  [12] , Cover  and  Leung  113)  have  found 
an  achievable  region  for  discrete  memory less  MAC'S  with  feedback  given 
by 

A - co[U  K(p)l  (1.2.10) 

peP 

vrtiere  fC(p)  la  given  by 

K(p)  - {(Rj^fBj)*  ^ KXj^jyIx^U) 

< I(X2>y1Xj^0)  (1.2.11) 

Rj^+Rj  < KXj^X^jY)} 

and  P Is  the  set  of  joint  probability  assignments  on  (DjXj^jX^)  for  which 

p(uXj^X2y)  - p(u)p(Xj^|u)p(X2|u)p(y|xj^X2)  (1.2.12) 

The  region  given  by  (1.2.7)  Is  at  least  as  large  as  that  given  by  (1.2.2) 
since  we  can  always  choose  0 to  be  a degenerate  randon  variable  which 
takes  on ‘only  one  value  with  probability  one.  In  irtilch  case  (1.2.7) 
reduces  to  (1.2.2). 

When  applied  to  the  noiseless  binary  erasure  MAC,  this  region 
can  be  shown  to  Include  the  point  given  by 

Rj^  ■ R^  * .7911  blts/transmlsslon  (1.2.13) 


A 
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An  outer  bound  based  on  total  cooperation  between  the  transmitters 
can  be  used  to  show  that  the  maximum  achievable  R for  which  R^^  =*  R^  = R 
is  given  by 


R * .7925  bits/transmission 


(1.2.14) 


The  result  obtained  by  Cover  and  I>eving  is  thus  very  close  to  optimum  for 
this  channel. 

In  [13] , Cover  euid  Leung  fovmd  an  achievable  region  for  the 
AHGN  MAC  with  feeedback  aned.ogous  to  that  specified  by  (1.2.7),  given 
by  the  set  of  all  (R^fR^)  such  that 


1 ^1^1 

R.  < i In  ( 1 + -i^) 

1 ^2^2 

R,  < i In  ( 1 + -^) 

2-2  O*  

1 P +P  +2/a  a P P 

R^  + R2ljln(l  + ■'12^ 


nats/transmission 


(1.2.15) 


where  and  ■ ^“®i  i ■ lr2.  In  figure  1.3  the  region  de- 

scribed by  (1.2.15)  is  superiiqposed  on  the  non-feedback  capeicity  region 
of  figure  1.2.  The  region  given  by  (1.2.15)  is  formally  identical  to 
that  given  by  (1.2.7)  trtien  all  the  random  variables  have  density  func- 
tions, power  constraints  are  i]ic>osed  on  the  Inputs,  and  a "test  encoder" 
of  the  form  shown  in  figure  1.4  is  enployed.  This  encoder  yields  the 
mutual  informations  appearing  in  (1.2.15),  and  the  joint  probability 
density  function  of  (U,X^,X2,Y)  is  of  the  form  given  in  (1.2.12). 
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1.3  Aciiievable  Region  of  Cover  and  Leung  for  the  ANGN  MAC  vdth  Feedback 
for  * ^2  “ 
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We  now  present  a heuristic  discussion  of  the  coding  approach 
of  [13]  and  of  the  encoder  of  figure  1.4.  The  essence  of  the  coding  approach 
is  that  during  a code  block  transmitter  1 (T^)  usesd  fraction  of  his 
power  to  trzmsmit  a "new"  message.  independently  chooses  a new  mes- 

sage, transmitted  by  a fraction  of  his  power.  Each  uses  the  remaining 
m of  his  power  to  transmit  information  known  to  both  transmitters , 

which  is  derived  from  past  data.  This  information  is  represented  by  random 
variable  0 in  fig.  1.4.  Using  the  feedback  link,  T^  ceui  both  perceive  what 
the  receiver  has,  euid  decode  the  new  code  word  sent  by  T^.  Since  U is  known 
by  both  treuismitters  and  T^  knows  his  own  transmissions,  the  rate  at  vdiich 
he  can  reliably  decode  T^'s  new  message  is  simply  the  mutual  information 
between  ' s input  emd  the  receiver ' s output  (which  is  also  T^ ' s output) , 
given  U wd  X^.  The  same  reasoning  applies  with  T^  amd  T^  exchanged. 
Requiring  then  that  each  tramsmitter  decode  the  other's  new  message,  we  have 
that 

“i^i 

R < i In  (1  +-^)  (1.2.161 

L ^ a* 

The  receiver,  however,  is  rather  more  confused  at  this  point,  as 
it  knows  neither  U nor  either  of  the  new  code  words  a priori.  We  now  define 
U to  be  an  encoded  signal  Intended  to  resolve  the  receiver's  uncertainty 
adxjut  the  "new"  messages  of  the  previous  block.  Since,  by  the  reasoning 
of  the  previous  paragraph,  each  transmitter  at  the  end  of  a block  knows  both 
codewords  and  the  state  of  the  receiver' s knowledge , the  transmitters  can 
cooperate  to  coherently  resolve  the  receiver's residual  uncertainty  about 
both  previous  messages.  Random  variable  U denotes  this  encoded  data. 
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Now  this  joint  codeword  is  designed  to  be  reliably  decodable  in  the 

presence  of  channel  noise,  and  while  treating  the  encoded  new  data  as 

noise.  Therefore  0 can  be  decoded  first  by  the  receiver  and  its  effect 

reBK>ved,  and  then  the  receiver  can  resolve  as  much  uncertainty  about  the 

new  Mssages  as  possible  from  the  remaining  signal. 

We  can  now  obtain  the  last  part  of  equation  (1.2.15) (the  first  part 

is  just  (1.2.16)).  We  note  that  w upper  bound  on  the  amount  of  uncerted.nty 

about  a massage  resolved  by  a single  use  of  a gausslem  chtmnel  is  given  by 
1 ^.T 

■T  ln(;^,  where  P_  is  the  total  received  power  , and  a*  is  the  power  of 
that  portion  of  the  received  signal  which  is  Independent  of  the  message. 

This  is  the  standard  capacity  result  for  gaussian  noise  channels,  with  the 
change  that  "unwanted”  transmitted  data  are  combined  with  channel  noise, 
we  now  consider  a block  of  data  and  the  two  new  mssages  associated  with  it. 
Since  that  portion  of  the  received  signal  which  corrects  old  data  is 
decoded  zmd  removed,  the  net  signal  received  is  just  channel  noise  plus 
the  twc  new  codewords,  %dLth  powers  *^^2*  these  variables 

are  independent  of  each  other,  the  total  energy  received  is  just 


Pt  - + oijPj  + a* 


(1.2.17) 


of  idiich  is  Independent  of  the  data.  Therefore  the  uncertainty 
removed  at  the  receiver  in  this  stage  is  bounded  by 


, o, P-  +a.P- 

»llj  ■ " I 


(1.2.18) 
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F\arther  uncertainty  is  resolved  by  the  U transmissions  in  the  next  block. 
Since  the  0 data  is  decoded  while  treating  the  sujpdrin^osed  codewords 
as  noise  the  effective  noise  for  this  code  is  given  by 


I*  - 


° * “l^l  * “2^2 


(1.2.19) 


The  effective  total  power  is  Just  the  total  received  power  given  by 


“ Vl  “2^2 


- Pi  + P2  + 


(1.2.20a) 


(1.2.20b) 


where  the  first  two  terms  in  (1.2.20a)  are  due  to  new  data,  the  third  term 
is  due  to  coherent  trMsmission  of  u,  and  the  fourth  term  is  additive  noise. 
The  amount  of  vmcertainty  resolved,  H^,  is  then  bounded  by  the  quantity 


..  , I ‘■t  1 , 

«-<  — In-^-  — In : 


‘2  — 2 “*  7^  2 


” *“i  W2 


(1.2.21) 


The  total  entropy  about  the  original  two  messages  which  can  be  resolved  at 
the  receiver,  which  bounds  the  total  data  rate  is  then 


\ * "2  i “1  * “2  i 2 


(1.2.22) 


which  is  the  last  part  of (1.2. 15) 

A rigorous  proof  of  the  achlevabllity  of  this  region,  based  on 
typical  sequence  arguments,  is  provided  in  [13]. 
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An  outer  bound  to  the  capacity  region  for  MAC'S  with  feedback 
can  be  obtained  by  generalizing  equation  (1.2.2)  to  encon5)ass  all  joint 
distributions  on  instead  of  just  statistically  independent  distrib- 

utions. Statistical  dependence  between  channel  letters  is  made  possible 
in  the  feedback  cause  since  the  transmitters  have  the  common  variadsle  rep- 
resented by  the  feedback  data  available,  on  which  to  base  their  new  trans- 
mitted letters.  Hie  authors  of  [13]  eiqilolted  this  possibility  implicit- 
ly by  having  the  transmitters  partially  cooperate  in  transmitting  their 
joint  message.  The  region  obtained,  while  laurger  than  the  non-feedback 
region,  is  strictly  smaller  thaui  the  outer  bound. 

In  general,  it  is  not  clear  how  the  transmitters  can  cooperate 
to  achieve  the  outer  bound,  or  whether  it  is  possible  in  all  causes.  In 
Chapter  2 below,  we  present  a method  for  the  AWCai  case  which  actually 
achieves  the  outer  bound. 

1.2.2  The  Broadcast  Chauinel 

The  broadcast  channel  was  introduced  by  Cover  [4] . Prior  to 
the  appearance  of  [4]  communication  with  more  than  one  receiver  waM 
generally  considered  in  the  context  of  time  or  frequency  division  multi- 
plexing, in  which  the  transmitter  aillots  paurt  of  its  totad.  power  to  each 
of  any  number  of  receivers  by  dividing  its  power,  in  either  time  or  fre- 
quency, between  signals  intended  for  each  receiver.  The  nature  of  the 
results  attadnable  using  this  approach  is  discussed  at  length  in  [5]  idiere 
it  is  shewn  that  the  su^zpositlon  approach  Introduced  in  [4]  allows 
sisniltaneous  conmninicatlon  at  rates  outside  the  region  allowed  by  various 
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sharing  strategies.  An  exception  to  the  vise  of  time  sharing  in  the  earlier 
literature  is  for  the  citse  of  two  users  commvinicating  with  each  other  over 
a common  channel,  discussed  in  a quite  general  setting  by  Shannon  [6] . 

Figure  l.Sa)  shows  the  general  two  user  broadcast  channel,  with 
dashed  lines  representing  feedback  links.  Figure  1.5b)  is  the  special 
case  of  physically  degraded  channels.  In  this  dissertation  we  treat  the 
problem  where  the  channels  are  discrete- time  additive  \riiite  gausslan 
noise  channels,  with  an  average  power  constraint  at  the  transmitter. 

In  order  to  obtain  a single  characterization  of  the  capacity 
region  of  a broadcut  channel,  it  has  so  far  (except  for  special  cases, 
e.g.  [27])  been  necessary  to  be  able  to  compare  the  channels  with  out- 
puts Y and  Z in  some  way.  Identifying  a channel  with  its  output,  we 
say  that  Y and  Z satisfy  one  of  the  following  relations  if  the  correspond- 
ing probabilistic  or  information  theoiretlc  relationship  is  satisfied. 

In  all  cases  below,  U and  X are  random  variables  for  which 

p(u,x,y,z)  - p(u)p(x|u)p(y,2|x)  (1.2.23) 

i)  Z is  a physically  degraded  version  of  Y if  p(y,z|x)  ■ p(z|y)p(y|x). 

That  is,  Z is  Independent  of  X given  Y. 

il)Z  is  a degraded  version  of  Y if  p(z|x)  • ^ p(c|y)p(y|x) . 

y 

ill)  Z is  more  noisy  than  Y if  I(n;Z)  ^ I(U|Y). 

Iv)  Z is  less  capable  than  Y if  I(X;Z)  < I(X|Y). 


I — , 

I 1 


1.5  Broadcast  Qiatmel  with  TVio  Receivers. 

a)  General  Case 

b)  FlVBically  Degraded  Case 
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Note  that  ii)  is  satisfied  if  and  only  if  there  exists  a chiuinel  of 
type  i)  with  the  same  marginal  conditional  probabilities  p(y|x)  and 
p(z|x) . 

Each  successive  ordering  can  be  shown  to  be  strictly  weaker 
than  the  one  preceeding  it  [1] . 

The  capacity  region  for  each  of  these  orderings  has  been  found 
in  the  absence  of  feedback.  For  orderings  i)  and  ii)  (forward  part  by 
Bergmans  [28]  converse  by  Gallager  [30] ) and  iii)  (Komer  and  Marton  [18] ) 
the  capacity  region  is  characterized  by  the  set  of  all  rate  triples 

where  corresponds  to  a common  message  intended  for  both 
receivers,  is  intended  for  Y and  is  intended  for  Z,  such  that 
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the  message  Intended  for  receiver  2,  and  then  decoding  his  own  message. 
This  approach  also  leads  to  the  definition  of  "degraded"  message  sets 
(Komer  and  Marton  [33]),  for  which  receiver  2's  message  is  required  to 
be  decodable  by  receiver  1,  in  which  case  the  capacity  region  (for  a 
general  BC)  is  given  by 


^ < i(x,y|u) 

Rq  £mln(l(0|Y),l(U;2)) 


(1.2.26) 


Of  course  under  this  restriction  R^  might  as  well  be  zero. 

The  capacity  region  for  the  ANGN  BC  has  also  been  found.  Cover 
[4]  found  an  achievable  regicxi  given  by  the  set  of  all  rates  such  that 

_ ^ 1 - . otP. 

R,  < :r  In  (1  + — ) 

^ ” a* 

1 nats/transmlsslon  (1.2.27) 

where  P is  the  power  constraint,  (T^  is  the  additive  noise  variance  at 
receiver  1 (CT|^a‘) , a is  between  0 and  1,  and  S - 1 - a.  Bergmans  [7] 
proved  the  converse.  Of  course,  by  extending  the  notion  of  degradedness 
appropriately,  using  integrals  instead  of  matrix  product,  it  is  true 
that  all  hWX  BC's  are  degraded.  As  in  the  case  of  the  MAC,  the  capacity 
region  for  the  AM6N  BC  is  formally  identical  to  that  for  discrete  BC's. 

The  only  previously  kno*m  feedback  result  for  BC's  is  El-Gamal's 
that  feedback  does  not  increase  the  capacity  of  physically  degraded  BC's 
(discrete  memoryless  [8]  and  ANGN  [16] ) . In  Chi^ter  3 we  present  a 
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cons  tzuctlve  coding  scheme  for  AMGN  BC's  which  enlarges  the  achievable 
region  for  cases  where  the  degradedness  is  non-physical.  This  is  the 
first  demonstration  of  the  fact  that  feedback  can  enlarge  the  capacity 
region  of  broadcast  channels.  We  will  also  <^tain  outer  bounds  on  the 
capacity  region  with  feedback,  both  for  discrete  memory less  and  AWGW 
BC's.  In  addition,  in  Appendix  D,  we  give  a new  outer  bound  on  the 
capacity  region  of  general  discrete  memory less  BC's  without  feedback 
which  is  tighter  than  previously  known  bounds. 

Figure  1.6  is  an  example  of  the  results  of  Chapter  3,  for 
“ 2P/o|  * 10,  and  channel  outputs  Y and  Z are  independent  of  each 
other  given  the  input.  Figure  1.6  Includes  the  time  sharing  line,  the 
superposition  curve  (equation  (1.2.27)),  and  the  a<diievable  region  and 
outer  bound  of  Chapter  3. 

1.2.3  The  coding  scheme  of  Schalkwijk  and  Kailath 

Schalkwljk  and  Kadlath  in  [2]  and  Schalkwijk  in  [3]  presented 
coding  schemes  for  the  AWGN  channel  with  power-limited  transmission  and 
noiseless  feedback.  Actually  two  schemes  were  given;  the  one  in  [2]  is 
suitable  for  infinite  bandwidth  channels,  and  the  one  in  [3]  for  band- 
limited  channels.  Both  schemes  are  actually  presented  in  a discrete- 
time framework,  but  their  bandwidth  requirements  can  be  obtained  from 
sanqpling  theorem  considerations.  The  scheme  discussed  here  is  that  of  [3], 
since  in  the  infinite  bandwidth  case,  simultaneous  communication  at  capacity 
can  be  achieved  for  the  MAC,  and  time-sharing  generates  the  entire  capacity 
region  for  the  broadcast  channel. 
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1.6  Ceqpaclty  Region  without  Feedbadc  and  Achievable  Region  and  Outer 
Bound  with  Feedhack  tar  P ■ 10,  o^  ■ 1 and  * 2. 


I 
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We  consider  the  problem  of  comnninicating  a random  vcuriable  0 

using  average-power  limited  signals  over  a discrete-time  AWGN  channel 

with  instantaneous  noiseless  feedback.  This  communication  model  is  sketched 

in  Fig.  1.7.  Although  the  noiseless  assunption  is  necess2u:y,  feedback  with 

delay  can  be  Incorporated  with  sli^t  modifications.  We  will  communicate 

9 via  a block  of  N channel  transmissions  Assxmie  that  after  the 

th 

k transmission  the  receiver  has  an  estimate  of  6 given  by 

\ - 6 + 5^  (1,2J^) 

where  is  a zero  mean  gaussian  r.v.  with  veuriance  aj^.  Assume  further 
that  the  transmitter  knows  Then,  at  the  k+1®*  time,  the  transmitter 

sends 


(1.2.29) 


Mote  that  since  power  constraint  is  satisfied. 

k+1 

The  reoelver's  k+1**  channel  output  is  corrtqpted  by  an 

independent  additive  zero  Man  gaussian  r.v.  with  variance  which  we 
call  denoting  the  received  value  by 


'k+1  ■ S+1  * *k+l 


(1.2.30) 


We  have  assumed  that  i*  instantaneously  available  to  the 


transmitter,  so  that  both  transmitter  and  reoaiver  can  form  the  receiver's 


1.7  Single-Source  Single-Destination  Oonnunication  with  Additive  Noise 
and  Feedback. 
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new  estimate,  given  by 


S _ S 

®k  Vl 


r. 


k+1 


0 + 5 


k+1 


(1.2.  31) 


readily  shown  to  be  the  maximum  likelihood  estimate  of  0 given 
0,.  and  r,_.,.  From  (1.2.28)  and  (1.2.31)  we  see  that  is  a linear 


■■k+1' 


combination  of  and  *nd  is  availeUsle  to  the  tremsmitter  for  the 

next  transmission. 


To  con^jute  the  variance  of  write 


*k+l  ” ^+1  * ■ 


^k+I^ 

k+1 
2 


^k+l^‘ 


'^k+l^k 


k+1 


Mow 


«k*Vi 

\ ■ 

and 

^+1  " ?+l  * ?+l 
■ p + a* 


(1.2.32) 


(1.2.33) 


(1.2  J4  ) 


(1.2  J5  ) 
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Ttierefore 


Vi  “ \ 


*0  ^ p+a 


2 ,k+i 


(1.2.36) 


To  compute  the  performance  of  this  scheme,  assume  that  6 cam 
take  on  any  one  of  ||el|  equally  spaced  values  in  the  interval 
the  spacing  between  adjacent  values  of  6 is  given  by 


0 - 1 


(1.2.37) 


After  N iterations  (a  code  block),  the  receiver's  estimate  is  given  by 


W-9  * « N-1 


(1.2.3S) 


and  the  message  is  decoded  as  the  closest  allowable  value  of  0. 

An  error  is  made  if  is  sufficiently  leurge  that  em  incor- 

rect value  of  9 is  closer  to  the  estimate  than  the  trxie  one.  This  can 
occur  only  if 


-ll  2 


(1.2.39) 


so  that  the  probability  of  error,  P^,  is  bounded  by 


>-r  1 


(1.2.40) 


The  inequality  comes  from  the  fact  that  when  the  true  value  of  S is  at  an 


end  point,  noise  which  drives  6 out  of  [-  does  not  cause  an  error. 
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Since  is  s sUm  of  jointly  gaussian  r.v.'s,  it  is  gausaiam.  Therefore 


12Q(- 


) 


2\fp 

where  Q(x)  is  the  tail  of  the  gaussian  pdf,  equal  to 

Substituting  (1.2.36)  into  (1.2.41)  yields 

,2.  N-1 


ir/ 


e " ^®^2  ( lleil-D/jr.  ^ ^ 


(1.2.41) 


O 9 

e^'/2^y 


(1.2.42) 


Since  N total  iterations  au:e  used,  if  we  define  R to  be  the 
transmission  rate  in  nats/channel  use,  then 


llell 


HR 


(1.2.43) 


In  addition  the  capacity  of  the  channel  is  given  by  the  well- 
known  result 


C . i 1„( 

2 a 


nats/  tramsmisslon 


(1.2.44) 


Substituting  (1.2.43)  in  (1.2.42)  we  obtain 


% PW*  1/2 


(1.2.45) 


Nr  nr 

(We  have  replaced  e -1  by  e which  decreases  the  argument.  Since  Q(*) 
is  a monotonic  decreasing  function,  this  v^per  bounds  P'  .) 

For  the  initial  transmission,  6 is  scaled  up  to  the  average 


transmitted  power  and  sent  without  coding.  That  is. 


nierefore 


(1.2.47) 


(1.2.48) 

(1.2.49) 


Subs'tltutlng  in  (1.2.45)  we  obtain 


P 

e 


(1.2.50) 


For  any  R < C the  «u:gument  can  be  made  arbitrarily  large  by 
increasing  N.  'Rius  since  Q(x)  decreeises  as  x increases,  the  error  prob- 

-X*  /2 

ability  can  be  made  arbitrarily  small.  Furthermore,  since  Q(x)  ~e 
andT^*-—  for  N large,  then 


%- 


3 P 2N(C-R). 


(1.2.51) 


ejdiibiting  a doubly  e]q>anential  decrease  with  block  length.  This  is  in 
contrast  with  non- feedback  codes,  ^ose  performance  is  typically  singly 
exponential  in  block  length. 
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An  additional  point  worth  mentioning  is  that  this  scheme  works 
for  arbitrary  zero  mean  finite  variance  noise  distributions,  as  long  as 
the  noise  is  uncorrelated  in  time,  and  independent  of  the  data.  The 
equations  giving  the  noise  variances  are  valid  regardless  of  the  gaussian 
sssuaiption,  although  the  eiqpressions  for  error  probability  are  no  longer 
necessarily  true.  In  [2]  it  is  shown  that  for  the  infinite  bandwidth 
channel,  the  scheme  analogous  to  the  one  used  here  does  yield  an  asyji{>totically 
gaussian  noise  term,  hence  doubly  exponential  behavior.  No  such  claim  is 
made  for  the  finite  bandwidth  scheme  of  [3]  (l.e.  the  one  discussed  here) . 
Singly  exqpcnential  decay  can  be  demonstrated  as  follows. 

**e  " (1.2.52) 


By  Chebyshev's  Inequality 


- p+a*  -2N(C-R) 

“IT* 


(1.2.53) 


Iherefore,  as  long  as  R <C,  p^  can  be  made  ajdiltrarlly  small. 

Of  course,  the  capacity  for  ncn-gausslan  channels  is  in  general  larger  than 
that  for  gaussian  cheuinels  with  the  same  variance,  so  the  set  of  rates 
achieved  by  this  scheme  is  sub-optimum  for  all  but  gaussian  channels. 


CHAPTER  2 


AWGN  MAC  WITH  FEEDBACK 


In  this  chapter  the  capacity  region  of  the  auiditive  white  gaussian 
noise  MAC  with  feedback  (fig.  2.1)  is  determined.  The  forward  part  of  the 
result  will  be  proved  constructively,  i.e.  a deterministic  coding  scheme 
(similar  to  the  one  in  Section  1.2.3)  will  be  presented  in  Section  2.1, 
yielding  an  achiev2d}le  region.  In  Section  2.2  an  outer  bound  to  the  capa- 
city region  is  found  by  meauis  of  a weak  converse.  In  Section  2.3  the  two 
regions  are  shown  to  be  equal. 

The  result  obtained  is  the  following. 


Theorem  2.1.  Define 


C - U {(R,  ,R,):  R-  < i ln(l  + — (l-p2)) 
OCp£l  12  12  ^2 


1 2 

Ro  1 T + -=-(l-p*)) 


(2.1.1) 


1 **1  * **2  * V^2lPl  , 

+ R,  < i ln(l  + — ) } 


2 - 2 


C is  the  capacity  region  of  the  AWGN  MAC  with  feedback,  vdiere  transmitter 
1 wd  2's  signals  (k  - 1,...,N,  i - 1,2)  satisfy 


-Lei”! 


(2.1.2a) 


I 


(2.1.2b) 


emd  the  additive  noise  has  variance  o? 
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2.1  AchieveOale  Region 

In  Section  2.1.1  we  demonstrate  the  achlevability  of  a certain 

rate  point  denoted  (R*,R*) . In  2.1.2,  superposition  arguments  au:e  used  to 

2 

extend  the  result  of  2.1.1  to  a non-en^ty  subset  of  R . 


2.1.1  The  Point  (R*,R*) 

Each  of  the  transmitters  i~l,2)  has  a message  6^  to  com- 

municate to  the  receiver  using  a code  of  block  length  N.  At  each  time  k 
the  transmitters  send  signals  over  the  forward  link.  These  signals  (real 
variables)  are  added,  corrupted  by  additive  gaussian  noise  with  zero  mean 
2uid  variance  a^,  and  received.  The  receiver  instantaneously  transmits  his 
received  variable  back  to  both  trjuismitters , so  that  the  k^^  received  veuri- 

able  y may  be  used  in  the  encoding  of  the  k+1®^  trcuismitted  data  x , , 

K l,k+l 


2Uld  X 


2,k+l’ 


We  assume  that  after  k-1  channel  uses  the  receiver  has  estimates 


of  9 and  0 of  the  form 


k-1 

" ^-1 


^-1 

0-0  + n 

2 2 'k-1 


(2.1.3) 


where  1 jointly  distributed  gaussian  random  variables,  with 

means  zero,  variances  and  b^  ^ respectively,  and  correlation  coeffi- 
cient F\irther  assume  that  the  estimates  are  formed  from  linear 
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I 


Combinations  of  previously  received  data.  Since  both  treuismitters  know  all 
of  the  previous  received  data,  then  knows  9^  . Since  he  also  knows 

then  knows  knows  2_*  Each  also  knows  the  covariance 

paurameters,  since  these  are  statistical  averages. 

At  the  k^^  step  each  transmitter  would  like  to  an^lify  his  cor- 
rection term  (i.e.  for  and  for  T^)  and  transmit  it  to  the 

receiver  as  in  the  scheme  of  Section  1.2.3.  In  general  these  terms  may  be 
negatively  correlated,  so  that  when  added,  they  can  destructively  inter- 
fere. However,  since  all  parties  involved  know  Pj^  one  trauismitter  can 
alter  the  sign  of  his  transmission,  so  that  the  trauismitted  signals  aire 
positively  correlated  axid  thus  constructively  interfere.  The  receiver 
can  then  use  its  knowledge  of  to  compensate  for  this  sign  change. 

We  require  then  that 


The  received  data  at  the  k^^  step  is  then 


(2.1.4) 


Ik 


+ X 


2k 


+ *. 


(2.1.5) 


where  is  the  new  AWGN  term  with  variance  a*,  independent  of  all  previous 
data. 
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Based  on  the  received  data  and  his  previous  estimates,  the  re- 
ceiver forms  new  estimates  of  9^  amd  given  by 


'Ic 

^ ^ 


'k  ^-1 


(2.1.6) 


Similaurly, 


e,  + 5.  , - - ■ ■ ^ r 

1 K-1  -T  ]( 


* V: 


Vk-x 


(2.1.7) 


It  is  easily  verified  that 


*ik  ■ 


^-x  ■'S;ir<'®r*'^ii>k-xi' 


(2.1.8) 


Vk-1  --^k 


-X  •’"“’k-x’"^*  '^l‘’k-ll  > 


By  defining  and  substituting,  we  obtain 


/ 


-45- 


^ - ^-1 


(2.1.9a) 


(2.1.9b) 


Similarly,  define  and  we  obtain 


c ■ c , 
k Jc-1 


!ki-i  Wi 


(2.1.9c) 


c. 

By  substituting  (2.1.8) , noting  that  p ■ — , euid  substituting  (2.1.5 


b. 


and  (2.1.9b)  into  (2.1.9c)  we  obtain 


3 - ’’"Vi  - 

* ''a"*Pj(l-pJ.j)  /cHPj 


(2.1.10) 


Let  us  assume  that  - p,  where  p e (0,1)  and  p solves  the  quartic 


equation 


P(P)  • O* (o^+P  +P,+2/P  P p ) 
12  12 


-(a*+p^(l-p2))  (a^+P^d-p^))  (2.1.11) 


We  will  show  in  Section  2.3  that  this  equation  also  specifies  the  joint 
*^^*^*^^^'*bion  of  *2^X2  ^bat  achieves  a point  on  the  boundary  of  the 
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outer  bound  to  the  capacity  region  of  the  AWOI  MAC  with  feedback,  and 
use  this  fact  to  determine  the  capacity  region.  In  Appendix  A it  is 
shown  that  P(p)'  has  exactly  one  root  in  (0,1).  By  squaring  (2.1.10) 
and  subtracting  from  1,  we  obtain 


(o*+Pj^  ) (^*+^2  ^^*Pk-l’  ^ 


(2.1.12) 


But  the  rational  term  equals  one  by  equation  (2.1.11).  Hence 
Pj  • P^.j^.^and  Ipj^I  - |Pjt_il.  In  (2.1.10)  note  that  if  0<pjj_^<l,  then 
Pj^  is  strictly  less  than  and  if  -l<p,^_j^<0,  then  pj^  is  strictly 

greater  than  Pj^_j^.  Therefore  if  IPj^.j^I  e (0,1),  then 


^k  “ “^k-l 


(2.1.13) 


The  rational  term  in  the  iterated  version  of  (2.1.12)  is  now  1,  so  that 
(2.1.13)  holds  inductively. 

We  now  describe  an  initialization  procedure  which  can  achieve 
any  desired  initial  correlation  (>*+1) . At  the  -1*^  epoch,  let  T^^  trans- 
mit 

• ^ 


1»-1 


'^*+1  ^ 


(2.1.14a) 


where  a is  a number  to  be  determined.  Let  T.  transmit 

2 


(2.1.14b) 


I 
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It  is  true,  then,  that 


-r  2a^  „ 

1,-1  a^+1  1 


(2.1.15a) 


2,-1  a*+l  2 


(2.1.15b) 


At  the  0 epoch  send 


X * ^ ^ --  - 0 

1*0  A^7i^^  ^ 


(2.1.16a) 


e 


(2.1.16b) 


It  is  easily  verified  that 


x:  1 + xf  ■ 2P, 

1,-1  1,0  1 


(2.1.17a) 


X,  1 + 3^  rt  “ 2P. 

2,-1  2,0  2 


(2.1.17b) 


so  that  the  average  power  constraints  are  satisfied. 
NOW  the  -1*^  ^md  0^^  received  data  are 


» 

i 
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Therefore , 


0 0 


(2.1.21) 


Any  value  of  p in  (0,1)  can  be  obtained  by  setting 


-1  .L. 


(2.1.22) 


Returning  to  our  problem,  we  choose  ot  so  as  to  obtain 


Po  = P 


(2.1.23) 


where  p solves  P(p)-0. 


Equations  (2.1.13),  (2.1.9)  and  (2.1.10)  then  become 


P^  - (-D-'P 


Vl 


b a b 
A 0 


a*  + P2(l-P*) 

a2+p^+p^+2j/ppj|p| 


r -t-  P^(l-p^)  ^ 

0 [^0*+P^+P2+2f5pj|p| 

Fa^tPid-p^)  T 

° [a*+P^+P2+2,/5]pJ|p|  _ 


(2.1.24) 


(2.1.25a) 


(2.1.25b) 
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Define 


R*  = i In  

+ P^d-p^) 


R*  - i In  i^-2 L_2 

O*  + P^(l-p^) 


-2kR* 


^ ^ -2kR* 

‘’k  ’ '’O  ® 2 


(2.1.26a} 


(2.1.26b) 


(2.1.27a) 


(2.1.27b) 


Proceeding  as  in  Section  1.2.3,  let  0^  equiprobably  take  on  one 
110^11  values  distributed  uniformly  between  -j  and  j.  Then  for  a block 
of  N transmissions  (corresponding  to  N-2  iterations  after  the  initialization) 


R.  ■ ^ In  I 1 0 . I I nats/transmission 
1 N 1 I 1 1 


(2.1.28) 


The  separation  between  adjacent  values  of  0^  is 


NR,  NR 

2(e  i - 1)  2e  i 


(2.1.29) 


^.1' 


the  error  probability  for  message  1 is  given  by 
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2e  1 /a^ 


(2.1.30) 


e 


N{R*-R^) 


) 


emd  similarly 


2e 


2R* 


(2.1.31) 


Both  of  these  can  be  made  arbitrarily  small  by  increasing  N,  so  long  as 
< RJ  for  i=l,2.  These  error  probabilities,  like  their  counterpart  for 
the  single  user  scheme  in  Section  1.2.3,  decay  to  zero  in  a doubly  expo- 
nential manner  with  increasing  block  length. 

By  way  of  a numerical  example,  consider  - 10,  and  * 1. 

Then  the  equation  P(p)«0  is  satisfied  by  p ■ .71164,  and 


RJ  * R^  = .8905  nats/tr£Uismisslon  (2.1.32) 


For  the  scheme  described  in  [13] , the  largest  value  of  R for 
which  Rj^  ■ R^  ■ R is  achievable  is  given  by 


R ■ J In (2  ^ 1 -1)  » .8643  nats/trauismission  (2.1.33) 


The  largest  achievable  value  without  feedback  is  (from  equation  (1.2.3)) 
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R “ ^ In  21  » .7611  nats/tremsmission  (2.1.34) 

2.1.2  Completion  of  the  Achievable  Region 

We  now  extend  the  result  of  Section  2.1.1.  We  will  describe  a 

superposition  approach  which  allows  communication  at  all  points  along  a 

curve  between  (R*,R*)  of  Section  2.1.1  wd  point  D of  Section  1.2  (see 

Fig.  1.2).  By  symmetrical  arguments,  the  point  (R*,R*)  and  point  C cam 

2 

be  connected.  By  using  the  fact  that  amy  point  in  R dominated  by  am 
achievable  point  is  in  turn  achievable,  we  will  have  demonstrated  the 
achievaOiility  of  a region  of  the  form  shown  in  Figure  2.2. 

We  allow  to  choose  two  independent  messages,  6^^^  amd  3^^^. 

will  transmit  by  means  of  a code  of  block  length  N whose  letters 

are  drawn  independently  from  a zero-meam  normal  distribution  with  variamce 
aPj^(0£a<l).  uses  the  remainder  of  his  power,  aP^(a=l-a)  to  participate 
in  the  scheme  of  Section  2.1.1  while  treating  the  code  letters  for  6^^^  as 
noise.  uses  all  of  his  power  as  in  Section  2.1.1. 

Since  the  code  letters  for  9^^^  au:e  independent  identically  dis- 
tributed N(0,aP^),  the  results  of  the  previous  section  apply  for  the  detec- 
to\ 

tion  of  9^  amd  9^,  except  that  a*is  replaced  by  a*  + aP^^,  amd  is 
replaced  by  ap^  everywhere.  Equation  (2.1.26)  then  becomes 

a^+oP.  -Ka*,  +P  +2/aP  P ‘ I p 

R*(a)  - i In  [ ± i— £ 

a*  + op^  + P^d-p*) 


1 


(2.1.35a) 
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^ o"+p +P IpI 
j In  ( = — = i-=- 

a^+op^+p^d-p^) 


, O^+OtP  +0®  +p  +2^  p 1 p 1 

R*(a)  - f in  ( 1__1  . 2 ^ I 2'  'j 

a^+aPj^+ocPj^d-p*) 

1 , .’’'♦wVSvJpI 

- 7 In  t ^1 

a^+Pj^d-ap*) 


(2.1.35b) 


The  equation  satisfied  by  |p|  (replacing  (2.1.11))  is  now 


P(a,p)-(a^+oiP^)  (a^+Pj^+P2+2^^p  |p|) 


- (a^+Pj^  (l-op*) ) (a^+ap^+p^  d-p*) ) 


(2.1.36) 


At  the  end  of  a block,  0^  and  are  Jcnotm  with  liigh  probability 
at  the  receiver,  so  that  all  of  the  correction  terms  are  also  )cnown.  These 
terms  can  be  subtracted  from  the  received  data  (which  must  be  stored) , 
leaving  just  the  codeword  for  0^^^  and  the  added  channel  noise.  This  code 
may  be  decoded  reliably  at  rates  up  to 


a^+oP, 


RJ*(a)  - j ln( pi-) 


(2.1.37) 


The  net  result  is  that  T^^  can  communicate  at  all  rates  satisfying 


< R*(o)  + RJ*(a) 


^ O^+P+P  +2^5^21  p aW, 

i In  [ J^-2 ^ ^ i ln(-^  ) 

o^+oP^+P^d-p*)  ^ 


and  T^  at  rates  satisfying 


(2.1.38) 


Rj  < m<x) 


a*+p^+p2+2  oPj^p^  I p I 

2 In  t ; :: 

o*+Pj^(l-ap^) 


(2.1.39) 


Since  p satisfies  equation  (2.1.36),  we  can  solve  for  the  denominator  term 
in  R*(a)  and  express  equation  (2.1.38)  as 


«!  < J In  [(l+^(l-ap*)] 


(2.1.40) 


Note  that  when  a - 0,  P(a,p)  reduces  to  equation  (2.1.11).  When 


a > 1,  P(a,p)  becomes 


p(l,p)  - (o*+Pj^)  (o*+Pj^+P2)-{a*+Pj^)  (a^+p^+p^d-p*))  (2.1.41) 


which  has  the  unique  solution  p « 0.  Denoting  the  solution  of  (2.1.11)  by 
p*,  noting  that  P(a,p)  is  continuous  in  both  p amd  a,  and  is  guareuiteed  (by 
Appendix  A)  to  have  a unique  root  in  (0,1),  we  see  that  as  a varies  between 
0 and  1,  p (and  op)  varies  continuously  between  p*  and  0,  so  that  the  right 
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hand  side  of  equations  (2.1.40)  and  (2.1.39)  veury  continuously.  At  a - 1, 
(2.1.40)  and  (2.1.39)  become 

1 ^1 

«!  < 2 (1+^1  (2.1.42a) 

1 ^2 

R2  < J In  ( 1 + ^7^  ] (2.1.42b) 

tdiich  is  just  point  D in  Figtire  1.2. 

We  can  estimate  (bound)  the  overall  probability  of  error  for  this 

12) 

approach  as  follows:  Let  e, «c  be  the  probabilities  that  6 emd  6 are 

12  12 

decoded  incorrectly.  By  the  results  of  Section  2.1.1  these  probeUsilities 
can  be  driven  to  zero  by  making  N large.  Let  be  the  probability  that 
0^^^  is  decoded  Incorrectly  given  that  6^^^  and  are  decoded  correctly* 
idiich  also  may  be  made  arbitreurily  small.  Then  by  the  union  bound 


Pr (error  on  6^^^  or  0^^^  or  02]  £ Pr[error  on  0^^^] 

(2) 

+ Pr[error  on  0^  ] + Pr[error  on  02l  (2.1.43) 

MOW* 

Pr [error  on  0^^^]  ■ Pr [error  on  02  correct] 

Pr[0^^^02  correct]  + Pr [error  on  0^^^(0^^^  or  0^  incorrect] x 
(2) 

Pr[0^  or  0^  incorrect] 
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U 


1 


(2)  (2) 

< e^d-PrEe^  or  0^  incorrect])  + Pr[0^  or  0^  incorrect] 

12) 

* + Prt0^  or  0^  incorrect] (l-e^) 

(2) 

£ + Pr[0j^  or  02  incorrect] 


< + €2  + (2.1.44) 

so  that  (2.1.43)  becomes 

Pr (error]  < 2e^  + 2e^  + (2.1.45) 

We  have  shovm  that  we  can  reliably  communicate  at  all  rates  eu:- 
bitrarily  close  to,  but  dominated  by,  a curve  in  R from  (R*,R*)  given  by 
equations  (2.1.26)  zuid  (2.1.11)  to  point  D in  Figurel.2.  By  reversing  the 
roles  of  and  on  this  section  we  caui  prove  a similar  result  for  a 
curve  between  (RJ,R*)  and  point  C in  Fig\irel.2.  The  entire  region  of  the 
form  displayed  in  Fig.  2.2  is  thus  achievable.  Doubly  eiqponential  error 
decay  is  no  longer  guaranteed,  since  the  error  probability  is  now  dominated 
by  the  error  rate  for  a non-feedback  code,  which  has  singly  exponential 
behavior. 

2.2  Converse 

In  this  section  we  obtain  an  outer  bound  to  the  capacity  region 


I 


1 


k 

I 


2.2  Capacity  Region  of  the  ANGN  MftC  with  Feedback  fba:  P,  » P.  = 10 
and  o*  - 1.  12 
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o£  the  AWGN  MAC  with  feedback.  In  2.2.1  we  give  a single  letter  charac- 
terization of  the  outer  bound  for  discrete  memory less  MAG's.  In  2.2.2  we 
argue  that  the  formal  result  of  2.2.1  may  be  applied  to  the  AWGN  MAC  with 
the  addition  of  a power  constraint,  cuid  evaluate  the  expression  using  stan- 
dard entropy  inequalities. 

2.2.1  Outer  Bound  for  Discrete  Memoryless  MAC'S 

Theorem  2.2.  The  capacity  region  of  a discrete  memory less  MAC  with  feedback 
is  included  in  the  region 

Cq  - CO  [ K(p)  ] (2.2.1) 

2 

where  cot*]  denotes  closure  of  the  convex  hull  (in  R ),  euid  Kip)  is  given 
by 

K(p)  - {(R^.R^):  R^  < I(J^j  Y |3^) 

R2<I<»2;y|Xi)  (2.2.2) 

Ri  + R2  < i{Xj^X2;y)} 

an4  P is  the  set  of  all  joint  probability  assignments  on 

The  difference  between  this  outer  bound  and  the  capacity  region 

of  the  MAC  without  feedback  is  that  the  union  is  carried  out  over  all  p(x  ,x  ) 

12 


rather  them  just  product  (i.e.,  independent)  distributions. 
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The  convex! fication  in«>lied  by  (2,2.1)  is  really  vmnecessary  since 

y((p)  is  already  convex.  To  see  this,  note  that  the  constraints  in  (2.2.2) 

are  concave  (Appendix  B)  and  consider  any  two  points  inUK(p).  If  they  are 

P 

contained  in  a single  K(p),  then  any  convex  combination  is  also  in  K(p) 
since  each  fC(p)  is  by  itself  convex.  Assume  then,  that  they  are  in  dif- 
ferent Kip),  say  (Rj^,R2) eK(Pj^)  and  (R^,R2) eK(p2) . Define 
11  2 2 

(Rj^,R2)  - a(Rj^,R2)  + (1-a)  (R^,R2)  euid  p » aPj^+(l-a)P2,  where  a e 10,1]. 

Then 


R 


1 


(2.2.3a) 

< ip{x^;Y|x2) 


R2  - aR2+(l-a)R2  1 

(2.2.3b) 

- aCRj+Rj)  + (l-aKR^+R?)  <al  (X  x ,Y)  + (X  X.;X) 

^ ^ ^ P2  i 2 

<I^(XiX2;Y)  (2.2.3c) 

where  the  subscripting  of  the  mutual  informations  denotes  evaluation  at  the 
appropriate  probability.  The  first  inequality  in  each  case  is  by  definition 
of  K(p^)  and  the  second  is  by  the  concavity  of  the  mutual  informations  as 
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functions  of  p(x  ,x  ).  Equation  (2.2.3)  shows  that  (R, ,R_)  e K(p)  C , 

X z 1,  2 O 

SO  that  is  convex.  This  reasoning  does  not  apply  to  the  MAC  without 
feedback,  since  a conxex  combination  of  product  probabilities  is  not 
necessarily  a prodtict  probability.  Therefore  convexlfication  is  necessary 
in  (1.2.1). 

We  proceed  to  prove  Theorem  2.1.  We  define  the  data  sources  to 
independently  produce  strings  of  letters  drawn  from  arbitrary  discrete 
alphabets  at  rates  (lail,2)  symbols  per  second,  we  denote  the  size  of 
the  i alphabeth  by  M^<«.  We  further  assume  that  the  sources  stationary, 
that  is,  the  joint  statistics  of  any  finite  string  are  independent  of  time 
origin.  For  message  a string  of  L letters  produced  by  source  i,  we  de- 
fine the  quantity 


**L^*i^  ■^H(a^)  nats/letter  (2.2.4) 

which  is  the  average  per-letter  entropy  of  source  i,  for  a string  of  !• 
letters,  in  general  a bounded  monotonically  decreasing  function 

of  L [24] . 


We  consider  the  problem  of  transmitting  data  produced  by  sources 

1 and  2 over  a channel  using  a block  code  of  length  N.  Assuming  the  channel 

may  be  used  H times  per  second,  then 
c 


(2.2.5) 


source  letters  axe  produced  per  code  block.  The  entropy  of  this  string  is 


given  by 
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H(a.)  * L H i(aj)  nats 
—i  T ^ i 


(2.2.6) 


and  the  amount  of  information  communicated  per  channel  use  (rate)  is  given 


M ® i(*j)  nats/transmission  (2.2.7) 


We  now  define  the  encoding  and  decoding  strategies  allowed.  For 
transmitter  i,  we  require  that 


2 )c-l,N 


(2.2.8) 


where  X is  the  i ” transmitter's  channel  symbol  at  time  k,  f.  is  a deter- 
If*  X 

k-1 

ministlc  function,  and  is  the  receiver's  channel  output  up  to  emd 

k-1 

including  time  k-1.  y is  available  to  both  transmitters  via  the  feed- 


back links. 


Slioilarly,  define  a pair  of  decoding  functions 


*1  “ i-l»2 


(2.2.9) 


which  are  functions  of  the  vector  of  N channel  outputs. 

We  will  use  a per-letter  form  of  Fano's  inequality  ([24],  Theorem 
4.3.2)  which  states  that 


In(M^-l)  + h(e^)  - n^(e^)  nats/letter  (2.2.10) 
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where 


and 


h(x)  » -X  In  X - (1-x)  In  (1-x) 


.th 


(2.2.11) 


(2.2.12) 


»nd  a ( or  S . . ) is  the  j letter  of  ^ (or  ^ ) . 
ij  ij  -i  -1 

Now  consider  source  1.  Since  apply  the  data 

processing  theorem  [24]  to  obtain 


- H(a^|a^j^) 


Also,  since  conditioning  cannot  increase  entropy. 


“(a.!  11*2^ 


Applying  (2.2.10)  we  have 


By  independence, 


Subtracting (2. 2. 15)  from  (2.2.16)  we  have 


(2.2.13) 


(2.2.14) 


(2.2.15) 


(2.2.16) 
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11*2^  ^ 


(2.2.17) 


We  bound  the  left  hauid  side  of  (2.2.17)  as  follows: 


I(ai»l|a2^  “ “<ll*2^  " “<ll*l£2^ 


I i2>  %i2>l 


I '<V' 


(2.2.18) 


Step  a)  is  true  since  X is  a deterministic  function  of  (Y  a^) , 

2k  — —2 

step  b)  is  true  since  conditioning  cannot  increase  entropy,  and  c)  follows 
from  the  fact  that  given  the  current  inputs,  the  output  is  independent  of 


the  past. 


We  can  combine  (2.2.17)  eind  (2.2.18)  to  obtain 


il''*2k’»kl’‘2k>  i "<il>  - 


(2.2.19) 


Subtracting  both  sides  frosi 


-64- 


“"i-j 


Note  that  — “ — (eqiiation  (2.2.5))  which  is  fixed,  so  that 
" **c 


(2 


JC*i  C 


(2 


A similar  series  of  steps,  applied  to  source  2,  yields 


'2  - 

k-1 


(2 


We  also  consider  the  joint  source  (a^,^^) . As  in  equation  (2.2.13) 


+ H (*2 1-1-2^ 


< H(a^|a^)  + H(a2li2) 


(2, 


so  that 


(2, 


The  left  hemd  side  is  bounded  by 


- H(Y)  - H(Y|aj^a2) 


.2.20) 


.2.21) 


.2.22) 


2.23) 


2.24) 


/ 
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- H(Yj[ 


.Jc-1 


a) 


b) 


<l  [H(Y.)  - H(Y.  |X  X a )] 


k«l 


k'  Ik  2k-  -1^' 


I [H(I^)  - 


k-1 


N 


I ''Vsk’^k’ 


k-1 


(2.2.25) 


where  a)  follows  since  conditioning  cannot  increase  entropy,  and  b)  follows 
since  given  inputs,  the  current  output  is  independent  of  the  past. 
Substituting  (2.2.25)  into  (2.2.24)  we  have 


) (2.2.26) 


Subtracting  from  N(R^  ^*^2^  " independence) 

we  get 

1 N 

N[Ri  + R2  " N ^^^lk^2k’\^^  - ^ (2.2.27) 


and 


"1  * *2  - ? Vxk’V  i ^ * ^2<"2> 


Now  any  code  of  <u:bltrary  block  length  must  yield  average  (per- 

txansmission)  mutual  informations  which  lie  inside  C . To  see  this,  assume 

o 

a block  length  of  N and  a joint  probability  assignment  on  k3^>k2 
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N 


puiiji 


t2.2.29) 


In  general,  the  probabilities  at  any  time  will  depend  on  the  values  at  all 
other  times,  but  the  mutual  informations  of  equations  (2.2.21)  (2.2.22)  2md 
(2.2.25)  depend  only  on  single-letter  probabilities,  so  this  dependence 
does  not  affect  the  average.  If  we  define  the  "average"  probability  assign- 
ment by 


- if  PJX  .Xj^) 


(2.2.30) 


k-1 


then  we  can  apply  Jensen's  inequality  to  (for  example)  I(x^;Y|X2)  to  obtain 


1 


(2.2.31) 


since  I(X^;y|x^)  is  concave  in  p(X^,X2).  All  of  the  mutual  informations 
are  concave  (Appendix  B) , so  that  (2.2.21),  (2.2.22)  and  (2.2.28)  become 


c 


’'2^^2) 

^ ^ "^2  - 

f c c 


(2.2.32a) 


(2.2.32b) 


(2.2.32c) 


tf^-vSTT 
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R 


Assume  (R^Rj)  is  not  in  C^.  Since  is  a closed  convex  subset 

2 

of  then  there  exists  a vector  (Xj^,A2)  where  A^,X2  > 0 and  X^,X^  are  not 
both  zero,  for  which 

1 Z O 

(see,  for  example  [26]).  in  peurticuleur,  there  is  a positive  6 such  that  for 

all  (x, ,x  ) eC 
i 2 o 


\®1  ^ V2  “ \\  ' ^2*2  > 6 > 0 


Combining  (2.2.32)  and  (2.2.34)  we  have 


L3 


’‘lir'ii'V  *^2r'’2<V 


Now  define 


0 < A 


0 < K 


e 

h*  (e) 


A 

M 


max(X  j^A2) 


< « 


< 00 


^ I hCe)  0 < e < j 

'1  J £ £ i 1 


(2.2.34) 


(2.2.35) 


(2.2.36) 


I 


i 
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and 


n(e)  - einCM^M^  - l)  + h' (e) 


(2.2.37) 


Clearly 


emd 


ri(e)  i max{nj^(e^)  ,n2te2^^ 


^ - ^1  iT”  ^2  iT”  ^2^^2' 

c c 

- ^^iT"  ^i^^i^  ■*■  kT  ^2  ^^2^^ 


(2.2.38) 


< XK[X^(e^)  + n2(e2^^ 


(2.2.39) 

(2.2.40) 

Since  n(0)  "0  and  Ti(e)  is  strictly  mononotonically  increasing  in  e,  then 
(2.2.40)  in^jlies  that  e is  greater  than  some  positive  constant,  independent 
of  block  length.  Theorem  2.1  is  proven. 

2.2.2  Outer  Bound  for  AWOi  MAC’S 

We  first  note  that  the  formal  result  of  Section  2.2.1  is  valid  for 
^hitrary  ensembles  as  long  as  the  various  entropies  Involved  can  be  defined 


< 2XKn(e) 


Therefore 


me)  > 5^  > 0 
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by  either  using  the  conventional  "eUssolute"  entropy  for  discrete  random 
vatriables,  or  the  "differential"  entropy  defined  by 

H(x)  - -/*  In  p(x)p(x)dx  (2.2.41) 

•00 

where  p(x)  is  a probability  density  function.  He  follow  the  standard  nota- 
tion (e.g.  [24])  in  using  the  same  symbol  for  absolute  and  differential  en- 
tropy. In  going  from  equation  (2.2.17)  to  (2.2.18)  we  identified  a mutual 
information  defined  as  a difference  between  edssolute  entropies  with  one 
defined  using  differential  entropies.  This  step  is  justified  so  long  as 
the  differential  entropy  is  defined  (i.e.  Y has  a density  function)  amd 
finite  ([24]  Section  2.5).  Y has  a density  ftinction,  since  it  is  a sum  of 
the  inputs  and  gaussi2m  noise.  Its  entropy  is  bounded  since 

- < i in  W - < H(X^|X^)  < H«^) 

1 9 

< j In  aj(y5^  + y^)  +a^) 

< « (2.2.42) 

Hhere  is  the  noise  veuriance  and  are  the  average  power  constraints. 

The  fourth  term  above  may  Involve  either  X or  X . . All  of  the  various 
conditioning  properties  hold  in  this  case. 

We  also  note  that  the  concavity  of  the  mutual  informations  in 
(2.2.2)  allows  us  to  reduce  the  per-bloclc  average  energy  constraint  to  a 
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per-letter  average  energy  constraint.  That  is,  for  any  code  satisfying 


I 


(2,2.43) 


a code  using  the  average  joint  letter  probability  law  (i.e. 

1 “ 

PCx^x^)  “ ^ I **^*l]c'*2k^  ^ yields  mutual  informations  which  are  no  smaller 
(when  averaged  over  the  block) , and  for  which  each  letter  satisfies 


""ik^^  k-1 N 


(2.2.44) 


We  proceed  by  fixing  a joint  probability  assignment  on  (X^/X^)  and 
bounding  K(p).  Any  assignment  on  (X^,X2)  has  the  characteristic  parameters 


“i  ■ *1  i ‘■i 


“2  ■ *2  i ‘'2 


XX 

0 • ^ C [-1,11 

12 


(2.2.45) 


We  have  assximed,  without  loss  of  generality,  that  X^^  and  X2  are  zero-mean. 
By  our  definition. 


Var(Y)  - E[Y  ] - + 


(2.2.46) 


where  o*  is  the  additive  noise  variance.  The  gaussian  distribution  has  the 
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maxijri?*  entropy  of  all  continuous  distributions  subject  to  a constraint  on 
the  variance.  Its  entropy,  for  vauriemce  s is  given  by 


j In  2ires  ^ g<s)  nats 


C2.2.47) 


The  entropy  of  Y is  then  bounded  by 


H(Y)  < g(Var(Y))  - g(aj  + a|  + 2aj^a2p  (2.2.48) 


Given  and  X^,  the  only  random  component  of  Y is  the  additive 
tdiich  is  gaussian,  so  that 


HCyIx^X^)  - g(a2) 


(2.2.49) 


Now  consider  the  conditional  entropy  of  Y given  X . By  definition 


H(y|X2)  " 


(2.2.50) 


where  the  expectation  is  over  X2.  Now  given  X2-X2  the  entropy  of  Y may  be 
bounded  using  (2.2.47),  by 


H(y|x  -X  ) < g(Var(Y|x  -X  )) 


(2.2.51) 


The  variance  of  y given  X.-x-  is  given  by 
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Var(Y|X2-X2)  - a*  + Var (X^ 


(2.2.52) 


since  channel  noise  is  independent  of  both  X^^  euid  The  second  term  on 

the  right  hand  side  of  (2.2.52)  in  turn,  is  by  definition 


Var(Xi|X2.X2)  - =xl^l*2“»2^>  IV*2J 


(2.2.53) 


where  the  ei^ectation  is  over  X^  given  X^^x^. 


Also  the  conditional  expectation  of  X^  given  ^2”^  '“'^‘3'*®  function 

of  x^  which  minimizes  the  right  hand  side.  To  upper  bound  the  expression, 
we  approximate  the  conditional  mean  by  the  best  linear  estimate  of  X^^  given 


X^^x^,  which  is 


‘1  ■ ^ <^2 


yielding 


V„(xJXj-Xj)  < E^t(x^  - ji  p Xjl^lXj) 


Averaging  over  X^  we  have 


Bjj^Var(X^|X2-X2)]  ^ Var(X^|X2) 


EKXj-^pXjl^l 


(2.2.54) 


(2.2.55) 
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5 A • 


aja-p^) 


(2,2.56) 


Going  back  to  equation  (2.2.50)  we  note  that  g(x)  is  concave  in  x, 
so  that  Jensen's  inequality  yields 

H(y|x  ) ^ E[H(Y|x  -x,)] 


< g(a*  + E(Var(x^|X2=X2)]) 


= g(a*  + Var(Xj^|X2)) 


< g(a*  + a^(i-p^)) 


(2.2.57a) 


since  g(x)  is  monotonic  in  x. 


We  can  exchange  x^^  and  X^  in  the  above  argtiment  to  obtain 


h(y|Xj^)  £g(a*  + a?(i-p^)) 


(2.2.57b) 


Substituting  (2.2.48,  (2.2.49)  and  (2,2,57)  into  the  definitions 
of  mutual  information,  and  observing  that 


g(x)  - g(y)  - i In  ^ 


(2.2.58) 


equation  (2.2.1)-(2.2.2)  becomes 


where 


Co  - 


K(P)  - {(Rj^,R2): 


R^  < j ln(l  + ^(l-p2)) 


R2  £ 7 + ^(1-p*) ) 


1 4.  2o,a-p 

^2-2  - ■ *2  ) 


P “ {pCXj^/X^)  : 1 


(2.2.59) 


(2.2.60) 


o\  < P_ 
2—2 


|P|  1 


(2.2.61) 


We  now  observe  that  a region  yielded  by  a negative  correlation  p 
is  included  in  the  corresponding  region  yielded  by  |p|.  Also,  for  positive 
p,  the  bounds  are  all  maximized  when  a*  - and  '^2'  Equation  (2.2.59)- 

(2.2.60)  may  be  re-written  then,  as 


Theorem  2.3.  An  outerbound  on  the  capacity  region  of  the  AWGN  MAC  with 
feedback  is  given  by 
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C - U K>  (p) 

0<p<l 


(2.2.62) 


where 


K'  (P)  * { < j Ind  + p-(l-p*) ) 


*2  1 J + ^(1-p^)) 


(2.2.63) 


1 ^ **2  2»^^P 

^)}  (2.2.63) 


We  have  replaced  K(p),  a function  of  a probability  assignment r by  fC' (p) , a 
function  of  the  correlation  coefficient. 

Observe  that  (2.2.63)  reduces  to  (1.2.3),  the  capacity  region  of  the 
AWGN  MAC  without  feedbacJc,  when  p * 0,  which  of  course  for  gaussian  (x^/X^) 
corresponds  to  independent  transmitted  signals. 


2-3  The  Capacity  Region  of  the  AWGN  MAC  With  Feedback 

We  will  now  show  that  the  achievetble  region  of  Section  2.1  and 
the  outer  bound  of  Tliieorem  2.3  are  identical,  to  simplify  notation  in  the 
sequel,  we  define  the  following  auxiliary  quantities: 


. . \ ^ * 2^|0| 
■ 1 + ' ■ ' 'J 


(2.3.1a) 


Pa^p>  - 1 + 5;ra-p*) 


P3(p)  - 1 + ^(l-p2) 


(2.3.2b) 


(2.3.3c) 


I 
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Equation  (2.2.63)  then  becomes 


K' (p)  * {(R^,R2) : 


R,  < jlnP^(p) 


®2  2 


*1  + ^2 1 i Rt^P^  ^ 


The  achieveible  region  is 


A - Aj^U  A2 


where  A.^  is  given  by  (2.1.39)  and  (2 .1,40)  , which  become 


(2.3.4) 


(2.3.5) 


^2  - I (2.3.6) 


where  p satisfies  (2.1.36),  which  we  repeat: 


P(a,p)  - (o^  + oiP^)P^(v^p)  - P^(^p){a2  + oP^+P^d-p*)) 


(2.3.7) 


is  given  by  an  identical  expression  with  P^^  and  P^,  R^^  and  R^,  and 
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s 

1 

i 


P,  euid  P„  reversed. 

A B 

Note  that  P^(p)  monotonically  decreasing  in 

I'p  0)  and  that  P^{p)  is  monotonically  increasing  in  p.  For  a particular 

value  of  p,  one  of  three  sit\iations  cem  obtain.  Either  — In  P + — In  P 

2 A 2 B 

is  less  than,  equal  to,  or  greater  them  J In  P^.  These  three  cases  and 
the  resulting  K* (p)  are  summarized  in  Figure  2.3. 

At  this  point  we  make  two  observations: 

I.  No  set  of  the  type  portrayed  in  Figure  2.3a)  can  contribute 
cuiy  points  to  that  are  not  in  sets  of  the  types  of  figures  2.3b)  or  c) . 

To  see  this,  recall  the  monotonicity  just  mentioned.  By  decreasing  p a small 
cunount,  Pj^(p)  and  Pg(p)  nay  be  Increased  incrementally  while  Pj(p)  decreases 
by  a sufficiently  small  amount  so  that  Figure  2.3b)  applies.  The  resulting 
K'  (P)  is  larger  them  the  original  one.  Thus  any  K'(p)  of  type  a)  is  strictly 
included  in  one  of  type  b) . We  will  use  this  fact  to  place  an  upper  bound 
(strictly  less  them  1)  on  the  value  of  p needed  to  generate  the  capacity 
region. 


II.  Any  point  on  the  boundary  of  must  be  on  the  boundary  of 
some  K' (p) . It  clearly  caumot  be  interior  to  any  K' (p) . That  it  is  ac- 
tually in  K’ (p)  for  some  p is  a consequence  of  the  fact  that  the  distance 

from  a boundary  point  to  C is  zero,  and  is  the  infimum  of  its  distance 

o 

from  K*(  p)  as  p vauries  on  [0,1].  Since  the  coordinates  defining  fC{  p)  aure 
continuous  functions  of  p,  the  distance  from  a point  to  IC(  p)  is  also  a 
continuous  function  of  p.  It  is  a standard  result  from  analysis  that  the 
extrema  of  a continuous  f\inction  on  a con^ct  set  aire  achieved  in  the  set. 
As  a result  of  this  observation,  we  see  that  every  point  on  the  boundaury  of 


I 

J 


I 

i 

i 

} 
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is  included  in  the  set  of  boundary  points  of  the  collection  of  K' (p) . 

We  now  find  the  point  on  the  boundary  of  C which  maximizes 

o 

+ R^,  and  show  that  it  is  just  the  point  (R*,R*)  of  Section  2.1.  The 
K'(p)  which  maximizes  R^^  + R^  must  be  of  the  type  shown  in  Figure  b) , since 
for  any  fC*(p)  : 


®2  - J I (2.3.8a) 

cuid 

+ R2  -i  (2.3.8b) 

The  first  bound  decreases  with  p;  the  second  increases.  The  mayirmim  must 

/ 

then  occur  when  they  are  equal.  This  equality,  however,  requires  that 

i In  P^(p)  + j In  Pg(p)  = j In  P^{p)  (2.3.9) 

or 

P;^(P)Pb^P^  = ^^(P)  (2.3.10) 

But  this  is  precisely  equation  (2.1.11)  (or  (2.1.36)  evaluated  at  a “ 0) , 
which  defines  the  correlation  coefficient  obtained  by  the  scheme  of  Section 
2.1.1.  Since  the  solution  to  (2.1.11)  is  unique  (Appendix  A) , our  coding 
schMM  yields  the  appropriate  correlation  between  X auid  X_.  The  point 

X w 

achieved  by  this  scheme  is  (from  (2.3.6)  with  a 0) 

«J  - i In  F^(p) 
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1 

R*  • ^ In 


'2  2 P^(p) 

A 


C2.3.11) 


By  substituting  P^(p)Pg(p)  for  Pj{p)  we  have 


R*  = 2 Pfi^PJ 


(2.3.12) 


so  that  (RJ/R*)  coincides  with  the  upper  right  hand  comer  of  jCv'p)  for  the 
p (which  we  will  call  p*)  which  yields  the  meucimum  {R^+R^) , cuid  thus  is 
that  extremizing  point. 

Now  consider  a point  on  the  boundeury  of  for  which  R^  ^ ^1* 
servation  II  above,  any  such  point  must  lie  on  the  boundary  of  K(p')  for 
some  p'.  Also  p'<  p*  since  P^tp)  is  strictly  monotonically  decreasing 
with  p (observation  I) . Finding  this  boundary  point  for  some  fixed  R^  is 
equivalent  to  meucimizing  R^  as  a function  of  rJ.  In  this  light  it  is  clear 
that  the  meucimum  (and  hence  the  boundary  point)  occurs  at  the  corner  labelled 

P in  Figure  2.3.c),  and  thus  that  p'  must  be  such  that  In  P (p')  * rI". 

2 A 1 

If  p'  were  larger,  then  R^  would  not  be  achievable  in  K(p'),  and  if  p' 
were  smaller,  then  P^(p')  is  diminished,  so  that  a smaller  R^  (bounded  by 
j In  P^(p')  - R^)  would  result. 

These  considerations  lead  us  to  conclude  that  the  boundary  of 
is  generated  by  the  comers  of  the  basic  sets  fC(P)  of  type  c)  as  p varies 
between  0 euid  P*.  At  P « 0 we  have  not  reached  the  positive  axes,  but  the 
time  sharing  argviment  applied  to  the  points 


■ I ^ 


1 ^ ^ ^2 
R2  - - ln(l  + 


C2.3.13) 


generated  at  p > 0 


“ 2 


R^-O 


(2.3.14) 


generated  by  not  using  Transmitter  2,  suffices  to  connect  with  the  axis, 

and  a similar  curgimient  connects  with  the  eucis. 

Now  a point  on  the  boundary  of  C for  which  R?"  > R*  is  generated 

o 11 

by  p'  < p*,  which  is  given  by 


rJ  * j ln(l  + ^(l-p'2) 


(2.3.15) 


The  value  of  R^  (call  RM  corresponding  to  this  value  of  p'. 


at  the  comer  P,  is  given  by 


®2  “ 2 ^ 


1 

2 ^ P^(p*) 


(2.3.16) 


Now  consider  region  A^.  From  equation  (2.3.6)  we  see  that  if  a 
p such  thatv/Sp  « p'  can  be  obtained  as  a solution  to  (2.3.7),  then 
(rJ,R2)  can  be  obtained.  As  noted  in  Section  2.1.2  at  a - 0, 
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y/ap  = p = p*,  and  at  a >•  1,  (obviously)  v/ap  « 0.  The  function  v^Sp  is  con- 
tinuous in  a cmd  p.  ThuS/  if  p (as  the  solution  of  (2.3.7))  is  a continuous 
function  of  a,  then  all  values  of  \/Sp  between  0 euid  p*  ceui  be  achieved  and, 
in  peurticular  any  (R^R^)  on  the  boundary  of  C for  which  R?"  > R*  is  at- 
tained  in  A^.  The  continuity  of  p as  a function  of  a is  a straightforward 
consequence  of  the  continuity  of  P(a,p)  in  both  a and  p,  and  the  fact  that 
a unique  root  exists  for  all  a. 

The  desired  result  (Theorem  2.1)  has  been  proven,  that 

Cq  = A^U  A^  (2.3.17) 

and  the  capacity  region  is  given  by  Cq.  We  refer  back  to  Figure  2.2  for  an 
exan^le  of  the  capacity  region. 

2.4  Capacity  Region  with  Correlated  Messages  and  Feedback 

In  [11]  SlepicUi  emd  Wolf  obtained  the  capacity  region  for  the 
discrete  memoryless  MAC  with  messages  which  have  a common  part.  That  is, 
in  addition  to  two  private  messages  and  a^,  known  only  by  T^^  euid  T^  res- 
pectively, there  is  a common  message  known  to  both.  Slepiem  and  Wolf 
showed,  defining 

Ri-^H(^)  i-0,1,2  (2.4.1) 


that  the  region 
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^C*  °°  (2.4.2) 

where 

R(P)  * {(8^/82):  Rj^  £ I(X^;y|X2U) 

R2  < I{X2;Y|Xj^U) 

Rl  + R2  1 I(XiX2;y|u) 

^0  ®2  I(Xj^X2;y)  ^ (2.4.3) 

and 

P=  {p(ux^X2y)  » p(u)p{x^|u)p(x2lu)p(y|x^X2) } (2.4.4) 

is  the  capacity  region.  (Actually  they  proved  the  we2dcer  result  that  outside 
^ is  bounded  away  from  zero  as  opposed  to  a per- letter  converse.) 
P is  the  set  of  joint  input-output  probadsilities  where  U ->•  {Xj^,X2)-^Y 
form  a Markov  chain  in  that  order,  and  (X^X2)  are  conditionally  independent 
given  u.  We  will  now  extend  the  results  of  Sections  2.1  and  2.2  appropriate- 
ly to  this  case.  We  will  show  that  if  P adaove  is  replaced  by  P'  where 

P*  - {p(uXj^X2y)  - p(u)p(x^X2|u)p(y|xj^X2)}  (2.4.5) 


that 


^c* 


(2.4.6) 
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is  an  outer  bound  to  the  capacity  region,  and  that  it  is  achiev2U3le  for 
gaussian  channels. 

To  derive  the  outer  bound  we  will  make  the  following  definition;  a^  and 
a^  are  as  in  Section  2.3,  amd  a^  is  the  common  message.  The  encoder  out- 
puts are  then 


-Uc  ■ 


(2.4.7a) 


(2.4.7b) 


Define 


"k  ■ 


(2.4.8) 


Now,  amalogous  to  equation  2.2.17,  we  have 


(2.4.9) 


Also 


N 


H(lla^)  - I H(yj^|Y^'^^) 


'-0-2 


k-1 


a) 


■ I “"kl 

k-1 


b) 


if'^kiVkij’ 


C) 


N 


< y H(Y  X„u  ) 
- r,  k'  2k  k 
k-1 


(2.4.10) 
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where  a)  is  true  when  is  a deterministic  function  of  *q»  *2^ ' 

b)  is  by  definition  of  U.^^,  and  c)  follows  from  the  fact  that  conditioning 
does  not  increase  entropy. 

Similarly, 


HliljoiA’  - I “"klx 

k«l 


I ■ Wl> 

k«l 


k»l 


■'SHV 


(2.4.11) 


where  a)  follows  from  the  deterministic  encoders  ^uld  b)  from  the  fact  that 
the  output  is  independent  of  the  past,  given  current  inputs. 

(2.4.10)  and  (2.4.11)  may  be  substituted  into  (2.4.9)  to  obtain 


(2.4.12) 


We  can  obtain  a similau:  expression  involving  iKa^)*  The  bound 
on  follows  by  similarly  showing  that 


k»l 


(2.4.13) 


and  the  bound  on  + ^2 


-86- 


.-Y)-  H(Y)  - H(Yla^,a-) 

—0—1—2  — — —'—0—1—2 

k-I 

All  of  the  siibsequent  steps  of  the  derivation  of  Section  3.2.1 

are  the  same.  The  sums  in  (2.4.12) > (2.4.13)  emd  (2.4.14)  can  be  reduced 

to  single  letter  quamtities  by  convexity  considerations  and  the  fact  that 

U -►  (X,,  ) .►  y is  a Markov  chain, 

k Ik  2k  k 

To  show  that  the  region  defined  by  (2.4.6)  is  achievable  for  the 
AWGN  case  we  use  the  following  superposition:  let  a^  be  mapped  into  a code- 
word of  block  length  N,  whose  letters  are  drawn  independent  identically  dis- 
tributed from  a gaussian  distribution  ('^(0,1).  Then  let  T^  amplify  the 
codeword  by  and  T by  /a-P,  where  0 £ a.  £ 1*  The  trauismitters  then 

use  the  remainder  of  their  powers  i=l,2)  to  cooperate  as  in  Section 

2.1,  where  the  correction  terms  are  computed  at  the  transmitters  by  assuming 
that  the  receiver  will  correctly  decode  At  the  end  of  a block  the  de- 

coder then  subtracts  the  superimposed  codeword  and  proceeds  as  in  Section 
2.1.2.  Clearly  this  scheme  can  achieve  emy  in  the  range 

0 1 Rq  - 2^"^^  * (2.4.15) 

wd  a derivation  similar  to  the  one  in  2.2.2  and  2.3  shows  that  the  inter- 
section of  the  achlevaUsle  region  with  wy  plane  of  the  form  R^  » R*  coincides 
with  the  intersection  of  the  outer  bound  with  that  plame. 
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CHAPTER  3 

AWGN  BROADCAST  CHANNEL  WITH  FWwnpAry 

In  tills  chapter  ws  introduce  a constructive  deterministic 
feedback  coding  scheme  for  the  AWGN  BC  with  feedback,  shown  in  Figure 
3.1.  The  noise  variables  are  Independent  of  each  other,  uncorrelated  in 
time,  and  independent  of  the  transmitted  data.  The  variance  of  Wj^  (the 
common  noise)  is  o*,  the  variance  of  (the  separate  noise  at  receiver 
i's  output)  is  a*.  We  assume  that  the  average  transmitted  power  is  i < mi 
to  HP  per  block  of  N signals.  Note  that  by  setting  (or  a*)  to  zero, 
our  model  includes  as  a special  case  the  AWGN  BC  with  independent  (or 
physically  degraded)  cheuinels. 

As  mentioned  in  Section  1.2.2  it  has  been  shown  that  for  both 
the  discrete  memoryless  [81  and  AWGN  [16]  BC,  feedback  does  not  enlarge 
the  capacity  region  if  6ne  channel  is  a physically  degraded  version  of 
the  other.  The  results  of  this  chapter  will  show  that  such  is  not  the 
case  when  the  channels  are  not  physically  degraded,  at  least  for  the 
AWGN  case.  Since  all  AWGN  broadcast  channels  are  degraded  (that  is, 
there  exists  a cascade  channel  with  the  sam  marginal  probability  density 
functions,  conditioned  on  the  input),  the  result  of  [8]  and  [16]  will  have 
been  shown  to  fail  to  apply  to  a class  of  (continuous)  degraded  channels. 

The  contents  of  this  chapter  parallel  those  of  Chaptei  ?. 

In  Section  3.1  we  present  the  feedback  coding  scheme,  and  obtain  an 
aj^ression  for  an  achievable  region.  In  Section  3.2  we  obtain  first 


3.1  AMGN  Broadcast  Oiannel  tidth  Feedback. 
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an  outer  bound  for  the  capacity  region  of  arbitrary  discrete  memoryless 
BC's  with  feedback,  and  then  extend  this  result  to  the  ANGN  case.  In 
Section  3.3  we  will  coopare  the  achievable  regicms  euid  outer  bounds  for 
some  examples. 


3.1  Achievable  Region 


For  the  problem  we  are  ccnslderlng,  a single  transmitter  wishes 
to  coBUBunlcate  two  messages,  0^^  and  S^,  to  two  separate  receivers  using 
the  feedback  channel  of  Figiire  3.1.  The  treuismltter  will  use  a single 
block  of  N transmissions  to  communicate  both  messziges.  We  assume  that 
the  feeedback  links  are  noiseless  and  delayless.  in  this  section  the 
outputs  and  in  Figure  3.1  are  denoted  by  r^j^  euid  r^j^  for  the  sake 
of  notatlcxial  conqpactness . 

■Assume  that  after  k-1  transmissions  the  receivers  have  their 
respective  estimates  of  6^  and  9^  in  the  form 


e 


k-1 

1 

k-1 

2 


(3.1.1) 


and  are  jointly  gaussian  with  zero  means,  variances  a^^  ^ and 
and  correlation  coefficient  As  in  the  case  of  the  MAC,  we  assume 

that  the  estimates  are  deterministic  functions  of  the  previously  received 
data,  and  are  therefore  available  to  the  transmitter.  Since  the  transmit- 
ter also  knows  the  messages,  it  can  form  ^ ^ and 


-90- 


Por  the  k transmission,  the  transmitter  forms  a linear  com- 
bination of  and  normalizes  it  to  average  power  P,  and  transmits 

it.  Let  the  transmitted  signal  be 


*lc  - Vf  ' ^ 9 sgn{Pj^_^)] 

v®k-l  v\-l 


(3.1.2) 


vdiere  V is  the  mean  squztred  value  of  the  term  in  square  brackets,  given 


P - 1 + g + 2g|p. 


(3.1.3) 


The  parameter  g e [0,<»)  may  be  varied  to  allot  a varying  proportion  of 
the  available  transmitter  power  to  communication  with  receiver  one  vs. 
receiver  two.  It  is  readily  seen  that  with  V chosen  ais  2d>ove,  the  va- 
riance of  Jtj^  (i.e.,  the  tremsmitted  power)  is  P. 

The  receivers  then  receive  noisy  versions  of  Xj^,  where  r^j^ 
(i*l,2),  which  is  the  i receiver's  channel  output  at  the  k^  time,  is 
given  by 


^ik  - *k  + ''k  + ''ik 


(3.1.4) 


%rtiere  w^^  is  the  common  noise  with  variance  a^,  and  w^j^  is  the  separate 
noise  at  receiver  i,  with  variance  o^. 

The  receivers  then  form  their  new  estimates  of  the  appropriate 
messages,  given  by 


i 


SJ  . 6,  * 5,  . oj-*  . %i- 


flk  . ok-l  *21c\-l 

2 2 T '2k 


Using  the  above,  it  is  eetsy  to  verify  that  for  i 


^ - p * a' t oj 


(3.1.5) 


(3.1.6) 


iSgn(Pjj_j^)  (g  + 


From  equation (3.1.5)  we  see  that 


r - r ^ik’^K-l 

^k  ^-1  ^ 


(3.1.7a) 


n - n ^2k\~l 

\ “ \-i =5 — '2 


(3.1.7b) 


These  may  be  squared  and  averaged  to  obtain  the  new  variances  given  by 

* •^-—px.P 


(3.1.8) 


Similarly,  if  we  define  Cj^  = and  T /then  we 

can  multiply  (3.1.7a)  and  {3.1.7b),  tcdce  expected  values,  and  obtain 

°k  “ Vl  ‘ =5“  4"  “ 4^-S^)  (3.1.9) 

'ik  '2k  'ik  '2k 

Also 


'lk'2k 


p + a* 


b.i.io) 


By  substituting 
°k  “ ’^k  Pk' 


(3.1.6)  and  (3.1.10)  into  (3.1.9)  and  using  the  definition 
we  obtain  the  following  recursive  ejqpresslon  for  Pj^; 


(a-Z^<T^a|)Pk,i-  IT  5 


pE 


(3.1.11) 


where 


E ■ P+o^+cJ+a* 

1 2 

n - (p+a*-KT^)  (p+a*+crf ) 

, , P 2 , (3.1.1 

"i  ■ ” 

"2  - V 

As  in  the  case  of  (2.1.10)  for  the  MAC,  equation  (3.1.11)  may 
be  shown  to  have  a solution  in  the  sense  that  there  exists  ape  [0,1] 
such  that  if  ■ p,  then  Pj^  ■ -p.  To  see  this,  one  can  substitute 
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L 


; j 

I 


! 


t 


Pjj  ■ “Pjc-i  (3.1.11)  and  square.  The  resulting  sixth-order  poly- 
nomial in  is  of  opposite  sign  at  and  Pj^.j^-l,  and  therefore 

has  at  least  one  solution  in  that  range.  Wfe  have  not  been  able  to 
determine  vdwther  the  resulting  vetlue  of  p is  unique.  The  values  of 
p for  the  numerical  examples  in  Section  3.3  were  obtained  by  iterating 
(3.1.11).  In  all  caises,  the  iterations  converged  to  an  appropriate 
value,  regau?dless  of  the  starting  point,  so  that  we  suspect  that  the 
solution  is  unique. 

Given  the  stable  value  of  p,  equations  (3.1.11)  and  (3.1.8) 

become 


P,,  - (-l)"p 


K ~ K 

k 0 


p+o*+o* 

^0*+o|+  5 (1-p*)"' 


(3.1.13) 


(3.1.14a) 


(3.1.14b) 


and  the  initialization  may  be  carried  out  in  a manner  analogous  to  that 
in  Section  2.1.  nie  error  probability  analysis  is  identical  to  that 
of  Section  2.1  and  is  not  repeated.  The  result  is  that  with  p selected 
as  the  (largest  if  not  unique)  stable  solution  of  (3.1.11),  then  all 
rates  such  that 


\ < »{(«)  - 7 In 


p+a*+o* 


<y*+oJ+  5 (l-P*) 


(3.1.15) 


I 


«2  ^ ^2^5’ 


, p+0*+a* 

L In __f 

cr*+a|+  ^ (1-p*) 


(3.1.15) 


2ure  achievable  with  error  probabilities  which  decay  doubly  exponentially 
with  block  length,  as  for  the  MAC. 

Die  solutions  obtained  in  this  section  are  evaluated  numerically 
for  some  exxuples  in  Section  3.3. 


3.2  Outer  Bound  to  the  Capacity  Region 


In  this  section  we  derive  a simple  expressicm  for  an  outer  bound 
to  the  capacity  region  of  the  broadcast  channel  with  feedback.  In 
Section  3.2.1  we  derive  a bound  for  discrete  memory less  channels,  and 
in  3.2.2  for  AHQ}  channels.  In  this  section  we  revert  to  the  notation  of 
Figure  3.1,  in  that  the  channel  outputs  are  denoted  by  Y and  Z. 


3.2.1  Outer  Bound  for  Discrete  Memoryless  BC's 


We  prove  the  following  result 


Theorem  3.1  Define 


Cl  - U {(Rj^,R2):  Rj^  < I(X;Y2|0) 
p£P 

Rj  ^ 1(0»Z)  } 

^2  "pVp^‘*l'V‘ 

Rj  ^ KXfYZIo)} 


(3.2.1) 


(3.2.2) 
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where 


P«  {p(u,x,y,z)  - p(u)p(x|u)p(y*|x)}  (3.2.3) 

Then  the  capacity  region  C of  the  discrete  aMBorylesa  BC  with  feedback 
satisfies 

CcC^nCj  (3.2.4) 

To  prove  this  result  we  introduce  the  channel  shown  in  Pigxire  3.2. 
Identical  to  that  of  Figure  1.5a)  except  that  receiver  one  knows 
receiver  two's  output.  Any  rate  pair  achievable  for  the  channel  of  Figure 
1.5a)  must  be  achlevedsle  for  that  of  Figure  3.2,  since  receiver  one  can 
always  Ignore  Z.  Therefore,  denoting  the  capacity  region  of  the  channel  in 
Figure  3.2  by  C , 

C C C (3.2.5) 

But  the  channel  of  Figure  3.2  is  physically  degraded,  since  if  we 
consider  (Y,Z)  to  be  receiver  one's  channel  output,  then  receiver  two's 
ou^ut,  Z,  is  trivially  Independent  of  the  channel  input  given  (T,Z) . 

The  result  of  [8]  then  appli*B»  that  is,  feedback  does  hot  enliurge 
the  civ»acity  region  in  Figure  3.2.  Therefore 

C'  - U {(R_,H,)t  R,  < l{X|YZ|o) 
peP  ^ ^ ^ ” 

Rj  < I{U}Z)  } 


(3.2.8) 
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• Cj^  (3.2.6) 

Therefore  Cc.C^.  By  reversing  the  direction  of  the  added  link  in  Figure 

3.2  and  exchanging  the  roles  of  receivers  one  and  two,  it  follows  that 
CCC^.  Equation  (3.2.4)  follows  immediately. 

In  i^^pendix  O we  apply  the  technique  of  this  section  to  generaO. 
BC's  without  feedback,  to  obtain  an  outer  bound  to  the  capacity  region  of 
those  channels. 

3.2.2  Outer  Bound  for  the  AWGS  BC  with  Feedback 

In  Section  2.2.2  we  applied  the  formal  result  of  Section  2.2.1 
directly  to  obtain  a single  letter  characterization  of  the  outer  bound 
on  the  capacity  region  of  the  AWGN  MAC  with  feedback.  We  used  the  fact 
that  the  differential  entropy  of  the  channel  output  was  well-defined  and 
bounded  and  that  the  resulting  Information  theoretic  constraints  were 
concave  in  the  input  probediillty  assignment.  For  broadcast  channels 
idiere  neither  channel  is  noiseless,  the  use  of  differential  entropies  is 
still  justified,  but  since  X(U;Z)  is  not  concave  in  p(u,x)  we  cannot 
directly  apply  the  single  letter  eiqpzesslon  of  Section  3.2.1.  In  [16], 
El-Gamal  obtained  the  nsult  that  feedbadc  does  not  increase  the  capacity 
of  physically  degraded  AWSN  BC's  using  a method  similar  to  that  used  by 
Bergmans  [7]  for  the  case  without  feedback.  Furthermore,  El*K;amal  showed 
that  in  the  case  of  ^ysical  degradadness,  allowing  the  better  receiver 
to  see  both  flannel  outputs  does  not  increase  ci^city,  regardless  of 
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whether  there  is  feedback. 


Using  the  results  of  [16] , we  will  prove  the  following: 


Theorem  3.2  For  the  channel  of  Figure  3.1,  with 


as  jL 

k-l  * 


(k»  1,...N) 


the  capacity  region  satisfies 


tdiere 


c c c^nc^ 


C - u {(R-,R,):  R.  < i In  (1  + ^ ) 

^ 0<O<l  A i Og 


Rj  < f-  In  (1  + 


ap+o*+a| 


C,  - U {(R,,1L):  R,  < I"  ln(l  + — ) 
* 0<pKl  ^ ^ 12  ClP+0*4C* 

R2<iln(l  + ^) 

e 


(3.2.7) 


(3.2.8) 


(3.2.9a) 


(3.2.9b) 


- a*+  ^ 


(3.2.10) 
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To  show  this  result  we  consider  the  ANON  version  of  Figxire  3.2. 
Assume  a coding  strategy  which  allows  reliable  cosnunication  at  some 
rate  padr.  Then  consider  the  problem  where  receiver  one's  output  is 


and  is  Invertible  {aa  long  as  , 

so  that  any  code  Involving  an  encoder  of  the  form 

\ - f(0j^,02,1^“^,z’'"^)  (3.2.12) 

and  decoders 


The  mapping  between 


- ( 


V 


(3.2.11) 


®1  " 


A 


(3.2.13) 


can  be  applied  to  the  new  channel  by  substituting  the  ^pprt^riate  fac- 
tional dependsnee  on  for  every  occurrence  of  The  code,  vrtien 

applied  to  the  new  diannel,  will  have  the  same  probadbility  of  error  as 
on  the  old  ohanael.  The  channels  represented  by  ( (Y,Z) ,Z)  and  ((S,Z),Z) 
axe  equivalent.  Ns  now  show  that  is  statistically  independent  of 
given  Sj^  so  that  the  fennel  (8,Z)  is  physically  degraded,  and  from  the 
result  la  [16],  has  the  sane  capacity  ((S,Z),Z)  with  feedback. 

nay  be  written  as  Z^^  > 


Maiden  variable  Z^^ 


'VV- 


i 


A 
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we  need  show  that  (Zj^  - Sj^)  is  statistic«illy  Independent  of  Xj^  given 


S.  . But 
k 


a^w  +a^w 

v®k  - ' > 


aj+a^  2k  Ik 


(3.2.14) 


Since  the  noises  axe  independent  of  the  signal,  is  Independent  of 

X^.  Since  the  noises  are  independent  of  each  other,  is  indepen- 


dent of  Wj^.  As  for  the  remainder  of  Sj^, 


r^icV  . >j 


(3.2.15) 


1 2 


Since  both  terms  in  (3.2.15)  are  gaussian,  and  they  are  uncorrelated, 
they  are  independent.  Therefore  is  independent  of  both  Xj^  and 

Sj^,  and  hence  independent  of  X^^  given  S^.  Z is  therefore  a physically 
degraded  version  of  S and  the  result  of  [16]  applies.  The  capacity 
region  desired  is  therefore  the  capacity  region  of  the  channel  with 
outputs  S and  Z with  no  feedback.  By  definition  of  S, 


®k  - *k  * \ + 


(y?w-.+o*w,. 

1 2k  2 Ik 


(3.2.16) 


so  that  the  channel  from  X to  S is  an  ANSN  channel  with  noise  variance 


a\al 


(3.2.17) 
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The  channel  from  X to  Z is  gau8si^m  with  noise  variance  a*+  a|.  Equation 
(1.2.27)  applies,  with  the  appropriate  noise  variances  inserted,  and  this 
yields  equation  (3.2.9a).  A symmetric  arg\iment  applies  in  obtaining 
(3.2.9b),  and  Theorem  3.2  follows. 

In  connection  with  the  AMGN  model  of  Figure  3.2,  we  have  obtained 
a coding  approach  suited  to  this  model  which  eiQ>lcys  sv^erposition,  made 
possible  in  this  case  since  receiver  one  cam  form  receiver  two's  estimate. 
This  coding  schesm  achieves  all  points  in  and  is  discussed  in  ^pendix 
C. 


3.3  Cosgarison  of  Inner  and  Outer  Bounds 

In  this  section  we  present  some  numerical  results  for  achiev- 
able and  converse  regions  for  various  realizations  of  Figure  3.1.  In  the 
subsections  to  follow,  we  will  discuss  degraded  channels.  Independent 
channels,  amd  scow  intermediate  cawes. 

3.3.1.  Degraded  Channels 

By  setting  ■ 0 in  Figure  3.1  we  obtain  the  physicatlly 
degraded  ANGN  BC.  For  this  model,  is  specified  by 

„ ^ 1 , . OP  . 

®i  1 2 m ’ 

(3.3.1) 

IL  < i In  (1  + ^ ) 

^ ^ ciP+c*+a| 
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which  is  just  the  standard  superposition  region.  C2  becomes 


< j In  (1  + ) 


OP 


ap+o‘ 


R2  <iln  (1  + ^) 


(3.3.2) 


The  region  described  by  (3.3.2)  is  the  tri2uigle  in  bounded  by  the 
positive  axes  and  a straight  line  from  (C^,0)  to  (0,C^),  i^ere 


C 


1 


(1  + 


(3.3.3) 


It  is  an  easy  matter  to  show  that  C^C  . 

In  Figure  3.3  we  plot  the  achievable  region  of  Section  3.1  and 
the  ctqpaclty  region  represented  by  (3.3.1)  for  P»10,  and  0^- 
nie  achievable  region  is  obtained  by  varying  g in  equation  (3.1.15),  and 
the  capacity  region  by  varying  a in  (3.3.2).  Observe  that  the  coding 
scheme  of  Section  3.1  is  sub-optimal  in  that  the  entire  capacity  region 
is  not  achieved,  nils  result  is  somewhat  disconcerting  in  view  of  the 
optimality  of  the  related  scheme  for  the  MAC,  u demonstrated  in  Section 
2.3. 

3.3.2  Channels  with  Independent  Moise 

By  setting  0 in  Figure  3.1,  we  obtain  a BC  with  Independent 
noises  at  the  receivers.  The  region  is  then  characterized  by 


A 


! 

1 
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3.3  Adilevable  RagLon  and  Capacity  Region  for  Physically  DegrzKied 
BC  %iith  Feedback  for  P » 10,  ct*  = 1 and  Oj  * 

t 

! 


I 
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<iln  (1+  f,) 
e 


(3.3.4) 


1 ^ In  (1  + 


OP 


OP  + a' 


vrtiere  - a^a|/(aj+cr|) . Ove  iregion  is  described  by  the  sane  expression 


with  1 and  2 exchanged. 

Unlike  the  physically  degraded  ceuse , where  C^C  , we  can  easily 

see  that  (OjC^)  emd  (C*,0)  aure  on  the  boundary  of  and  (Cj^,0)and  (0,C*) 

are  on  the  boundary  ofC^-G^is  the  single  user  capacity  to  the  1^^  receiver, 

2uid  C*  Is  the  capacity  to  a receiver  with  noise  veuriance  o^.  Since  for 

e 

and  a*  >0,  a*  < min(a^,a2),  then  (C*,0)  is  outside  of  and  (0,C*)  is 
outside  of  so  that  neither  region  Includes  the  other.  Figures  3.4 
to  3.6  show  the  achievable  regions  and  outer  bounds  for  equeil  noise  cases, 
for  P/0^  « P/O^  ■ 1,10,  and  100,  respectively.  Note  that  the  achievable 
regions  and  outer  bounds  are  quite  close  for  P/o^  = 10  and  100,  but  that 
this  is  not  the  case  for  low  signal  to  noise  ratios.  In  all  of  these 
cases  the  superposition  region,  which  represents  the  non- feedback  capacity 
region,  is  just  the  tine  sharing  region,  eis  can  be  seen  frcxn  the  super- 
position equations  fdiich  beccne,  for  * ^2  * ^c' 


®2 


OP+C* 

c 


(3.3.5) 


(1  + 


I 
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I 

3.5  Achievable  Region  and  Outer  Bound  for  AMGM  BC  with  Independent 
Noiaes  and  FeedBback  for  P > 10,  ^2 


i 

i 


3.6  Achievable  Region  and  Outer  Bomd  for  AMGM  BC  with  Independent 
Noises  and  Feedback  for  P ■ 100, 
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Addlng,  we  obt£d.n 


+ Rj  1 J * §2)  “ Cj^  ■ Cj  (3.3.6) 

c 

Since  (C. ,0)  Is  achieved  by  comminicating  with  only  receiver  one,  and 

• ■*> 

\ 

(OfC^)  by  ccomunicating  with  only  receiver  two,  tine  sharing  can  achieve 
any  point  on  the  boundary  of  the  region  represented  by  (3.3.6).  Thus, 
for  these  examples,  the  use  of  feedback  has  enlarged  the  capacity  regicxi. 

Figures  3.7  through  3.9  are  analogous  to  Figures  3.4  through 
3.6,  but  for  the  cases  where  a|  * * 1,10,  and  100.  The 

time  sharing  line,  and  the  boundary  of  the  capacity  region  without 
feedback  (superposition  curve)  are  added  to  each  of  these  exaaples,  as 
well.  Behavior  is  similar  to  that  for  equal  noise,  we  point  out  that 
thou^  the  marginal  noise  distributions  for  this  model  are  the  same  as 
for  the  degraded  exanple  of  Secti^  3.3.1,  the  achievable  region  here 
lies  outside  the  superposition  region,  vdiile  for  the  degraded  exaa^le, 
the  achievable  region  was  inside  the  sui>exposition  region.  The  behavior 
of  the  coding  scheme  of  Section  3.1,  therefore  depends  critically  on  the 
joint  statistics  of  the  noise  variables,  rather  than  just  the  marginals. 

Observe  that  in  Figure  3.6  the  achievable  region  is  not  convex, 
and  that  in  Figure  3.9  part  of  the  boundary  of  the  achievable  region  lies 


inside  the  auperposltlon  region.  The  coding  scheme  is  sub-optimal  for 
AxennlAe  as  well . 


-112- 


3.3.3  Intermediate  CMes 

we  conclude  this  section  on  numerical  results  with  two  inter- 
mediate cases,  that  is,  exan^les  where  the  noises  aure  peurtially  cor- 
related. Recall  that  for  the  examples  of  Figures  3.3  and  3.8,  the  total 
noise  variance  at  receiver  one  wSs  1,  and  at  receiver  two  it  was  2.  In 
the  former  case  the  ccmnon  noise  had  vauriance  1,  and  in  the  latter,  variance 
0 (it  wais  non-existent).  In  both  cases  P *10.  Figure  3.10  repeats  the 
results  for  those  cases,  and  gives  the  achievable  region  and  outer  bound 
when  the  ccomon  noise  has  variamce  As  elected,  the  results  lie  bet- 
ween the  previous  results. 

For  the  equail  noise  case,  a degraded  channel  is  the  degenerate 
case,  that  is,  o|*0.  In  this  case  both  receivers  are  the  same,  and  the 
capacity  region,  with  or  without  feedback  is  just  the  time  sharing  region. 
Figure  3.5,  therefore,  tdiich  had  the  resxilts  for  P/o*  * P/o|  ■ 10,  with 
no  conmon  noise,  as  well  as  the  time  sharing  line,  already  contains  the 
results  for  the  two  extreme  cases.  Figure  3.11  repeats  3.5  with  the  addi- 
tion of  the  outer  bound  for  <3^  * y.  For  this  set  of  parameters,  the 
achievable  region  was  virtually  Indistinguishable  from  the  outer  bound,  and 
could  not  be  plotted  separately. 
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3.U  Adiievsyi^  and  Otter  Bounds  fac  fMStf  BC  with  Feedback. 

P - 10  and  a*  + a*  - a*  + ai  - 1.  Ocsnon  Holse  Variance  o*  - 
0,  .5,  1.  ^ ^ 
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CHAPTER  4 
COHCLUSIOMS 

Vfe  have  presented  two  similu  feedback  coding  schemes  for  the 
discrete-time  AS9GN  multiple  access  and  broadcast  channels  respectively, 
niese  schemes  are  extensions  of  the  approach  given  for  the  band-limited 
single  user  ANGN  channel  with  feedback  by  Schalkwljk  and  Kallath  [2] , [3] . 
Ihe  multi-user  schemes  share  with  the  schemes  of  12]  and  [3]  the  properties 
that  they  axe  deterministic  and  achieve  doubly  e^qponential  decay  of  error 
probability  with  block  length,  although  for  the  MAC  this  holds  (xily 
for  rates  dominated  by  those  achieved  without  the  use  of  the  super- 
position approach  described  in  Section  2.1.2.  The  sets  of  achievable 
rates  afforded  by  these  schemes  exceed  previously  known  achievable 
regions,  except  in  the  case  of  the  physically  degraded  broadcaist  channel, 
for  which  the  true  capacity  region  was  already  known. 

We  have  also  established  outer  bounds  for  the  achievable 
regions  of  the  MAC  and  BC  with  feedback,  both  for  discrete  memoryless 
channels  and  for  the  ANQI  case.  The  outer  bound  for  the  ANOl  MAC  coincides 
with  the  achievable  region,  yielding  the  capacity  region.  For  the 
AM^  broadcast  channel  the  achievable  region  falls  to  reach  the  outer 
bound  in  all  cases.  This  failure  is  due,  to  at  least  some  extent,  to  a 
sub-<9tlmality  of  the  achievable  region;  the  coding  scheme  falls  to  achieve 
even  the  known  ci^acity  region  of  the  physically  degraded  AWSN  BC.  It  has 
not  been  determined  whether  the  outer  bound  for  the  BC  is  itself  too  loose. 
The  results  obtained  do  demonstrate  a previously  uneiqpected  fact:  feedback 
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can  enlarge  the  capacity  region  of  broeidcast  channels  as  well  aa  multiple 
access  channels. 

A number  of  areas  for  further  reseiurch  are  suggested  by  the 
results  of  this  dissertation: 

J 

1)  Extensions  of  the  general  approach  to  other  two  user  channels  with 
feedback. 

2)  An  achievable  region  or  outer  bound  for  M user  ^annels,  where 
M > 2,  for  either  the  MAC  or  BC  with  feedback. 

3)  Application  to  discrete  memory less  ch2umels.  This  work  depended  i 

on  the  existence  of  an  c^tlmal  c«istructlve  approach  for  the 

single  user  channel  and  on  the  fkct  that  transmitted  signals  combine  lin- 
early. A constructive  approach  for  single  user  erasure  channels 
Is  well  known  [241,  and  approaches  for  BSC's  ([20], [29])  and  for 
general  discrete  memoryless  channels  [21]  have  appeiured  In  the 
literature.  In  addition,  as  mentioned  In  Section  2.1,  a coding 
scheme  for  discrete  memory less  MAC'S  with  feedback  has  appeared 
[13] . One  question  of  particular  interest  is  iidiether  the  outer 
bound  for  MAC'S  with  feedback  found  In  Section  2.2.1  is  achievable 

4 

in  general. 

4)  The  gaussian  broadcast  diannel.  nie  results  of  Chapter  3 are  ’ | 


inooaplete  in  that  the  capacity  region  has  not  been  found.  From 
the  examples  of  Section  3.3  it  is  clear  that  the  achievable  region 
of  Section  3.1  is  not  the  ci^acity  region.  It  seems  doubtful  that 
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the  addition  of  supaxposition  followed  by  convex! fication  would 
be  optimal,  though  the  regions  so  obtained  might  in  general  lie 
close  to  the  outer  bound.  Conversely,  it  is  unclear  how  tight  the 
outer  bound  of  Section  3.2  really  is. 

5)  Noisy  feedback.  Xn  [2]  and  [3]  and  subsequently  ([22], [23]) 
attempts  were  made  to  analyze  and  optimize  the  basic  Schalkwi  jk- 
Kailath  approach  when  additive  noise  appeared  in  the  feedback 
link.  Under  the  reuonable  assumption  that  both  forward  and  feed- 
back powers  were  limited,  the  results  were  discouraging:  none  of 
the  approaches  were  able  to  yield  reliable  coonnunicatlon  at  auiy 
positive  rate.  Prom  a purely  Shannon  theoretic  point  of  view 
this  is  somewhat  beside  the  point,  since  the  existence  of  codes 
which  do  not  employ  feedback  is  gueuranteed  by  the  coding  theorem. 
Noiseless  feedback  simplified  the  coding  problem,  but  did  not  add 
anything  in  the  way  of  achievable  rates.  In  the  multi-user  case, 
however,  feedback  generally  does  enlarge  the  capacity  region,  in 
addition  to  allowing  a simple  coding  procedure.  Noise  in  the 
feedback  links  must  therefore  affect  the  size  of  the  capacity 
region,  and  as  the  feedback  links  become  totally  noisy,  capacity 
regions  must  degenerate  to  their  non-feedback  values.  Results  in 
neither  the  forward  nor  converse  directions  exist  as  yet. 

Two  classes  of  problems  related  to  that  of  noisy  feedback  are 


that  idiore  the  feedback  links  are  of  a different  type  than  the 
forward  links  (e.g.  gaussian  forward  links  with  noisy  or  noiseless 


-118- 


dlscrete  feedback) , and  the  case  where  only  one  feedback  link 
is  available.  For  the  MAC  this  would  correspond  to  allowing  the 
receiver  feed  back  to  only  one  transmitter.  In  the  BC  either 
only  one  receiver  could  feed  back,  or  perhaps  the  receivers  would 
share  a single  feedback  channel  in  a multiple  access  mode. 
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APPENDIX  Ki  ROOTS  OF  P(p) 


We  prove  that  the  qazurtlc  polynomial 


P(p)  - cr*  (o*+Pj^+P2+2/p^2P)  “ 

(o*+Pj^  (1-p* ) ) ( o^+P^  (1-p* ) ) 


(A.1) 


has  exactly  one  root  in  the  interval  0<p<l. 

The  following  are  true: 

1.  A quartic  has  at  most  four  real  roots. 

2.  Lim  P(p)  a — 00 

p4O0 

3.  P(0)  - -Pj^P^  < 0. 

4.  P(l)  - > 0. 

5.  P(-l)  - 0. 

3.  and  5.  imply  that  there  is  at  least  one  negative  root. 

2.  and  4.  ia^ly  that  there  is  at  least  one  root  greater  than  1. 

Hence  there  are  (from  1.)  at  most  two  roots  in  (0,1).  But  by  3.  and  4. 
there  are  an  odd  number  of  roots  in  (0,1).  Hence  there  is  exactly  one. 
Since  P^,  and  axe  all  arbitrary,  this  result  holds  also  for  the 
polynoadal  P(a,p)  defined  by  equation  (2.1.36). 


I 
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APPENDIX  B;  CONCAVITY  OF  MDTUAL  INFORMATION  FOR  THE  MAC 


In  this  appendix  we  show  that  the  quantities  I(X  X ;!),  I(x, ;Y|x  ), 
and  I(X2;y1Xj^)  are  concave  functions  of  the  joint  probability  assignment  of 
Xj^  and  Xj. 

The  first  quantity  is  concave,  since  it  is  just  the  mutual  in- 
formation between  the  input  and  the  output  of  a channel  (see  [24] ) . We 
need  only  show  the  result  for  KXj^jYjx^),  since  the  same  demonstration 
holds  for  KX^fXlXj^)  with  X^^  and  X^  exchanged. 

Consider  two  input  probability  eussignments  Pj^(Xj^,X2)  euid  p^CXj^jX^). 


Define 


^3^*1'*2^  - cq>j^(Xj^,x2)  + (l-a)p2(Xj^,x2)  a e[0,l]  (B.l) 


Obvioxisly,  p^  is  also  a joint  probability  assignment  on  X^^jX^.  Now  define 
the  auxiliary  random  variable  i,  with  probability  law  Pr[^  1]  « a,  amd 
Pr[**  2]  ■ 1-a. 

Consider  the  following  comiunlcation  scheam:  we  wish  to  communicate 
the  value  of  4 to  the  receiver.  A saa^le  of  9 is  drawn,  and  a corresponding 
X^fX^  are  drawn  from  the  joint  probability  assignment  corresponding  to  the 
sample  value.  That  la  (X^.X^)  has  probability  assignment  p^(x^,X2)  whan 
4 > 1.  Denoting  the  mutual  Information  resulting  from  p^  by  I^(X^;y{X2)f 
we  have  that 


I(Xj^;Y(X2«)  - Pr[*-llI^(Xj^;Y|X2)  + Pr [4-2112  (Xj^;Y|X2) 
- alj^(Xj^;Y|X2)  + (l-a)l2(Xj^lY|X2) 
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j The  mutual  information  unconditioned  on  $ is  just  I(Xj^;y|x2) 

evaluated  at  p^.  in  other  words, 

KX^fYlx^)  - l3(X^,Y|X2)  (B.3) 

Subtracting  (B.2)  from  (B.3),  we  have  that 

i{Xj^;y|x2)  - i(Xj^;ylx2f)  “ I3  - - (1-a) (B.4) 

, The  left  hand  side  is 

I 

I 

V • 

I iHdlx^)  - H(y|x^X2)J-lH(Y|X2$)  - HCyIXj^X^^)] 

- hcyIx^)  - H(y1x2») 

• « l{y;$lx  ) 

t 0 (B.5) 

where  the  first  equality  in  (B.5)  holds  since  given  Xj^,X2,  Y is  independent 
, of  4.  Substituting  (B.5)  into  (B.4),  we  have 
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APPENDIX  C: 

CODING  FOR  THE  AUGMENTED  AWGN  BC  WITH  FEEDBACK 


We  consider  the  model  of  the  BC  shown  in  Figure  3.2.  This 
model  is  an  extension  of  the  can<xiical  broadcast  channel,  in  that  one 
receiver  is  allowed  to  see  both  channel  outputs.  This  model  may  be  ap- 
propriate in  some  applications , say  in  cases  where  one  receiver  is  located 
^ysically  between  the  transmitter  and  the  other  receiver,  and  received 
data  is  fed  back  to  the  transmitter  directionally.  The  example  is 
interesting  theoretically  since  it  allows  the  e3q>loitation  of  the  super- 
position idea,  and  obtains  results  which  are  optimum  (i.e.  reliable 
coanunication  is  possible  at  all  rates  in  the  capacity  region) . The 
approach  of  Section  3.1  does  not  use  superposition,  and  is  subc^timal,  at 
least  for  the  only  case  in  which  it  was  applied  and  the  capacity  region 
was  known  (degraded  channel) . 

Recall  from  Section  3.2  that  the  capacity  region  for  the  AWGN 
version  of  Figure  3.2  is  given  by 


(Cl) 


where 


Wi  now  present 

at  all  points  satisfying 


a coding  schema 

(C.l) . 


which  achieves  reliable  CGsmMnicatlan 


\ 

i 

I 

% 

s 

j 

i 

i 

t 
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All  definitions  are  as  in  Section  3.1,  except  that  after  k-1 
transmissions  receiver  1 has 

* ^-182  ■ 61  * 8,.-i82  ♦ Sit-1 

(C.2) 

^ k-1 

® 2 - 02  + \-l 


and  receiver  2 heis  the  same  estimate  of  ^ above  is  assumed  to 

be  a known  constant,  since  receiver  1 knows  receiver  2's  channel  data, 

there  is  no  problem  in  his  forming  the  same  estimate  of 

th 

Now  for  the  k transmission,  the  transmitter  sends 


rP~  , ^-1  ^k-1 


{C.3) 


where 


V m I + 


(C.4) 


Receiver  1 obtains  r^^  and  r^^^,  while  receiver  2 obtains  r^j^,  where 


c,,.  • *.-  + w.  + w.. 
ik  K k ik 


(C.5) 


At  this  point  both  receivers  upgrade  their  estimates  of  62  as  before, 
foxsdng 


9?  . e'-i  - Szi- 

2 2 —5  2k 

r 


(C.6) 


Assume  that  jH  0.  Then 


To  upgrade  his  estinate  of  e^,  receiver  1 first  conibines  r^^  and 
r^j^  by  forming  the  maxianaB  likelihood  estiaate  of 

r'  - " X,.  + w.  + X.  + w,  + wJ 

^ K k _2.^  k k k 

(C.IO) 

He  then  subtracts  ^ (62  frcm  rj^.  The  terms  cancel,  yielding 

'3k’  VF  ^ ®3**k”'k  I®-!” 


>^-1 


(G.ll) 


Treating  6^  as  an  unknown  (but  non-random)  parameter,  then 


(C.12) 


'3k^-l  " V?  ’^-1 


(C.13) 
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Receiver  1 now  forms 


01  V2  " 0^^  ^ Vl02‘  '3k 

'3k 


01  ^ V2  * 5k 


(C.14) 


By  substituting  equation  (C.ll)  above,  defining  r^j^,  it  is  seen  that 


•^k  “ Vl  ^ 


Vi 

**k-l  p 


(C.15) 


P , 

r - r fi  ^ X ^ *^^-1 

5k  5k-i<l  “ P 4. j ^ P 4.  j 

V*0^e  V*  (i 


which  may  be  squared  juid  averaged  to  obtain 


<*11  ♦ *i> 


. . -n  . ^ I f * » Vi*^ 


Vl  P 0 

17+ 


(C.16) 


(C.17) 


We  can  multiply  equations  (C.  8 ) and  (C.16)  and  average  to 
the  following  recursive  eaqpressicn  for  the  correlation  between  the  estimate 
noises i 


^ *2)?V-1  ^3k5k-l  'iii^k-l  '3k  Vl 

-k-lVl  -r  -TV  2k^3k  


T*  r* 

'2k  3k 


*2kVl*2k5k-l 


(C.18) 
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By  asstmption  “ 0.  This  in^lles  that  ■ 0,  so 

that  the  first  amd  third  terms  of  (C.18)  are  zero.  From  the  definition 
of  .io«,  ...  that  w that  a.fl«ln9  T - 

for  i « 2 and  3,  we  obtain 


( - 1) 


(C.19) 


But  from  (C.  5 ) and  (C.ll)  we  obtain 


r_.  r..  ■ — + (w.  + w,' ) (w,  + W-.  ) 
2k  3k  p k k k 2k 

I <^*<^2 
■r< 


(C.20) 


Substituting  into  (C.19),  than, 

TO  inlti^llM,  the  Mssages  are  sent  separately 
(amplified  vqp  to  e^vected  power  P) , so  that 


- w_  4W  • 

®l  *0®2  " ®1  * 


(C.21) 


0 2 ,0 
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- 0 

*0  “ 12P 

0^^ 


C.22) 


Thus  the  hypothesis  that  « 0 Is  true  induction,  and 


ji^r 

^p+o*+of  J 


(C.23a) 


(C.23b) 


A siailar  analysis  as  in  Sections  2.1  and  3.1  yields  that 
reliable  cosnunication  is  achieved  at  all  and  such  that 

1 S+  o* 

*1 


«C.23c) 


(C.24) 


, P +0*  of 

sine.  V - 1.9‘,  V ell,-] , nd  ^ e(0,11 . Mtlelng  a - 1,  o.  obtein  th. 
desired  result. 


\ 
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APPENDIX  D;  AN  OOTER  BOUND  FOR  BROADCAST  CHANNELS 

D.l  Preliminaries  emd  an  Alternate  Representation  of  Prior  Results 

In  Chapter  3 we  found  it  desirable  to  obtain  an  outer  bound  on 
the  capacity  region  of  the  broadcast  channel  with  feedback.  In  this 
appendix,  we  apply  the  methods  of  Section  3.2  to  bound  the  capacity  region 
of  channels  without  feedback.  The  best  such  bound  for  general  broadcast 
channels  is  that  found  by  Sato  [31]  (also  in  (11).  For  a general  BC  with 
input  X and  outputs  Y and  Z,  with  marginal  conditional  probabilities  Ef(y|x) 
and  p*  (zjx),  let  p be  the  following  class  of  joint  channel  probability 
functions 


P“  {p(ys|x):  ^p(yslx)  - p1(z|x) 

y 

^p(y»lx)  - pXy|x)} 


(D.l) 


Then  for  each  pep  define 

ai 

R(p  ) - co[  U{  («,»»-)* 

qe(i  ^ 

Rj  <_I^(X;Z)  <D.2) 

«i  ♦ *2  1 iq(x,YZ)  Ip  1 


where  is  the  set  of  all  possible  input  distributions.  Sato's  outer 
bound  is  then 


■pcf 


(D.3) 
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In  equation  (D.2)  and  throughout  this  appendix  it  is  convenient 

to  express  Z(X;YZ)  as  I^(X>Y2)|p  to  denote  dependence  on  p,  the  channel 

probability  function,  and  q,  the  input  probability  assignment.  Similarly 

I(X}Y)  and  I(X;Z)  are  written  as  I (X;Y)  and  z (X;Z)  to  denote  their 

<1 

dependence  on  q.  They  are  Independent  of  p for  all  p e P. 

nie  inequalities  %dilch  define  P'!p)  in  (D.2)  follow  from  stan- 
dard information  theoretic  arguments,  and  the  intersection  in  (D.3) 
holds  since  a code  for  the  actual  channel  must  be  a code  for  every  channel 
with  the  same  marginal  conditional  probabilities. 

Zf  we  define  the  followii>g  quantity: 

J (X;YZ)  - inf  I (X»YZ) I (D.4) 

we  obtain  an  alternate  representation  of  (D.2)-{D.3).  Defining  the 
region 


C*  - co[  u * 

qed 


Rj  ± lq(X;Z) 


we  will  prove 

Bieorem  D.l  C*  " C 


(D.5) 


(D.6) 


I 

I 


Before  proving  Theorem  D.l  we  introduce  the  following  lemnata: 
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Leona  D.l  J (XiYZ)  is  concave  in  a. 

q 

This  follows  from  Theorem  5.5  of  [26]. 

leMga  D»2  For  R{p)  defined  as  in  (D.2),  and  for  euiy  non-negative  vector 
^ in  define 


M(^)  « SUp[Xj^l^(X;Y)  + X^l^CX.-Z)  + XjIqCXjYZ)  |p]  (0.7) 


Then  a point  (R*,R*)is  in  R(p)  if  and  only  if  for  all  ^ ^ 0, 


XlRj  + XjRj  * X3(RJ+R*)  < M(^) 


(D.8) 


The  above  applies  to  the  set  C*  with  J subsitituted  for  I (X;yZ)  • in 

q p 


(D.7) 


This  lemma  follows  from  an  application  of  the  separating  hyperplane 
theorem  (Theorem  11.3  of  [26]).  We  note  here  that  the  convexiflcatlon 
in  (D.2)  and  (D.5)  is  redundant,  since  both  underlying  unions  are  already 
convex  (since  all  of  the  constraints  eure  concave  and  is  convex) . 


D.3  Since  Z_(X;YZ)  , is  bounded  and  continuous  in  p and  q,  convex 

q |p 


in  p and  concave  in  q,  and  p and  are  convex  and  compact,  then 


inf  sup  I (X»YZ)|  - s\^  inf  I (X/YZ). 

pq**  IP  qp^  I** 


(D.9) 


This  is  Lesu  37.3.2  of  [26].  Note  that  the  continuity  and  cc^actness 
hypotheses  imply  that  sequential  limits  of  informations  are  actually  achieved. 


n I 
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We  new  prove  Theorem  D.l.  If  is  in  c*,  then  there  is  some 

q for  which  the  three  inequalities  in  (0.5)  are  satisfied  (agedn,  by 

continuity  and  oon5>actness,  (R*,R*)  is  not  only  approached  as  a limit,  but 

actually  achieved),  but  since  for  all  p,  Z (X;YZ) i > j (X;YZ),  the  same 

^ IP  — <1 

q yields  informations  which  satisfy  the  three  inequalities  in  (0.2)  for  all 
p.  Therefore  C*CC*. 

Conversely,  assume  (R*,R*)  'ls  in  C.  From  Lemma  0.2,  it  is  true 
that  for  «ai  p £ ',  and  for  all  X > 0, 

^1^*1  * £ sup[Xj^I^(X;y)  + X2lq(X;Z)  + 

q 

X3lq(X;Y2)  1^1  (0.10) 

Since  (0.10)  holds  for  all  pe  P,  we  can  take  the  infimvon  of  the  right  hand 
side  over  p,  and  apply  Lemma  0.3  to  obtain 


XiRJ  + Xj®!  + [X^lq(X;Y)  + X2lq{X;Z)  + 

**  ^ X,I  (XfYZ),  ] (0.11) 

3 q |p 


Only  the  last  term  in  (0.11)  depends  on  p,  and  its  infimum  is  J . 

q 

Therefore 


^1*^  + XjRj  + XjdlJ+Rj)  £SUP  IX^I^j(X;Y)  + X2lq(X;Z)  + 

^ X,J  (X»YZ)  1 (0.12) 

3 q 


A 


and  by 


D.2,  (l^,lt*)  is  in  O.  Therefore  C'CC*,  and  Theorem  0.1  is 
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proved. 


D.2  A New  Outer  Bound 

ffe  now  give  a new  outer  bound  on  the  edacity  region  of  broad- 
cast channels,  bued  on  the  approach  of  Section  3.2.  Consider  the  aug- 
mented cheumel  of  Figure  3.2 without  feedback.  The  capacity  region  of  this 
channel  can  be  no  smaller  than  that  of  the  corresponding  BC  without  the 
link  between  receiver  2 and  receiver  1.  The  augmented  channel  is 
degraded,  so  that  it  heus  capacity  region 


^ < I (0;Z)  } 


(D.13) 


where 

!• 

d*  ■ {q(u,x,y,z)  ■ q(u)q(x|u)p(yz|x)}  (D.14) 


The  ci^clty  region  C of  the  true  channel  must  lie  in  Cj^(p) 
for  all  p e *o  that 

c c n c,(p)  (D.15) 

PcP 

By  reversing  the  direction  of  the  added  link,  we  get  . I 

I 
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c c n C2{p) 

pep 


(D.16) 


urtiere 


C,(P)  - U {(R,,R,):  R,  < 

qe  a'  ^ ^ 

< Iq(X,Y2|0)  } 


(D.17) 


Therefore,  we  have  the  following 


Theorem  D.2  The  capacity  region  of  the  general  discrete  memoxryless 
BC  satisfied 


c c [ n c,  (p)  1 n [ n c,  (p)  i 

PeP  PeP 


n iq(p)nc2(p)i 

peP 


(D.18) 


where  P is  defined  by  (D.l). 


D.3  Comparisoo  of  the  Bounds 


We  will  prove  the  following 


Theorem  D.3  The  bound  of  Theorem  D.2  is  tighter  than  that  of  Theorem 
D.l.  That  is 


n[C,<p)nC,(p)]  C C* 


(D.19) 
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To  show  this,  w6  first  find  an  outer  bound  for  Pot  any  P»  if 

e Cj^tp)*  then  there  exists  a q for  which 


Rl  < lu)  Ip 

Since  0-*'X-*-(y,Z)  is  a markov  chain  (by  definition  of  then  ^ the 

data  processing  theorem  ([241  Theorem  4.3.3) 


{D29) 


Also, 


R_  < I (U;Z)  < I (X;Z) 
2 — q — q 


Rj^  + Rj  1 I^(X;Yzlo)  Ip  + lq(D,Z) 

< I (X;YZIU)|  + I (U,YZ)| 
~ <J  Ip  q Ip 

- Ip 

- Ip 


ID.21) 


(D.22) 


Therefore , 


Rj  lq(X;Z)  } 


(D.23) 


Using  an  i^proach  similar  to  that  which  proved  Theorra  D.l,  we  can  show 
that 
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nc,(p)ccf-  u {fR, /R,) » R,  + R,  < J„(X;YZ) 
pep  ^ ^ qca  ^ ^ <D.24) 

< I^(X|Z)  > 

Nto  can  define  c°  analogously,  and  obtain 

peP  tc^(p)nCj(p)i  c cjnc®  (d.25) 

Wb  now  show  that  show  this,  we  use  the 

equivalent  to  I^imia  D.2,  that  a cmvex  set  Is  cos^letely  specified 
by  the  set  of  Its  tangent  hypexplanes  (l.e.  equals  the  Intersection  of 
the  half-spaces  determined  by  the  hypeiplanes) , and  show  that  every  tan- 
gent to  C**  Is  tangent  to  ^2.^  ^2'  ^ '***  * geosietrlc  atxgument,  similar 

to  that  of  Section  2.3.  Recall  that  C*  Is  defined  by 

C*  - U {(R.pR-):  R,  < I (X;y) 
qcQ  ^2  1 - q 

®2  (D.26) 

Rj^  + Rj  < J^(X;YZ)} 

For  every  q,  the  set  defined  by  the  Inequalities  of  (D.26)  Is 
of  one  of  the  types  shown  In  Figure  0.1.  a slsdlar  situation  occurred 
lo  Section  2.3,  and  there  we  were  able  to  disregard  sets  of  the  type 
shown  In  Figure  D.la).  Here  we  can  show  that  type  a)  does  not  even  occur. 
Consider  that  channel  In  P for  which  T and  Z are  Independent  given  X. 
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This  channel  Is  always  in  P.  Call  this  channel  p^.  For  any  q, 

J^(X,Y2)  - inf  I^(X;YZ)  < I^(X,YZ)  (vSTJ) 

Since,  for  Y and  Z are  independent  given  X,  it  is  true  that 

< Hq(Y)  + H^(Z)  - Hq(y|x)  - H^(Z|X) 

- I^(X;y)  + I^(X;Z)  (D.28) 

where  we  have  dropped  the  dependence  on  p^  for  quantities  'irtiich  depend  only 
on  the  marginal  conditional  probability  functions.  Coobining  (D.2’7)  and 
(D.28)  we  have  that 

Jq(X,YZ)  < lq(X,Y)  + lq(X,Z)  (0.29) 

Therefore  sets  of  the  type  depicted  in  Figure  D.la),  irtiich  correspond 
to  the  reverse  (strict)  inequality,  do  not  occur. 

Mow  consider  a tangent  to  C*.  From  continuity  and  compactness, 
it  must  be  tangent  to  one  of  the  sets  of  Figure  D.l.  Assume  the  tangent 
has  equation  ■ k^(A},  where  0 ^ ^ £.1.  Wie  point  of  tangency 

must  be  the  point  marked  P in  Figure  D.l,  since  the  line  has  a slcpe  less 
than  -1.  HOW  assume  that  this  line  is  not  tangent  to  C®.  The  basic  sets 
of  are  the  same  as  those  of  Figure  D.l,  except  that  the  horisontal 
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llne  corresponding  to  the  constraint  on  Is  absent.  If  the  tangent  we 
are  considering  Is  not  tangent  to  assume  that  there  Is  a tangent  with 
the  same  slope  and  a larger  constant,  say  ^^(X)  > k^(X).  The  case  of 
a smaller  constant  cannot  occur,  since  C*C  C^*  + XI^  “ 

k^CX)  Is  tangent  to  a basic  set  of  some  other  q at  the  point  cor- 

responding to  P.  We  can  evaluate  the  basic  set  of  C*  Tor  the  new  q.  The 
new  line  Is  tangent  to  this  set  at  point  P,  since  the  liiq>osltlon  of  a 
constraint  on  will  not  delete  that  point,  because  of  Inequality  (D.29). 
Therefore,  there  Is  a point  of  C*  lying  above  the  assumed  tangent,  leading 
to  a contradiction.  Hence,  all  tangents  of  the  form  R^  + XB^  ~ ^(X)#  for 
0 < X < X Are  tangent  to 

In  a similar  fashion,  tangents  of  the  form  XH^  -i-  R2  ■■  k(X)  for 
0 < X < X o®n  be  shown  to  be  tangent  to  C°. 

Fran  these  two  facts.  It  Is  clear  that  C*  Is  no  smaller  than 

o o 

^Xnce  each  of  the  latter  sets  Includes  C*«  C*  Is  no  larger  than 

the  Intersection.  Therefore  Theorem  0.3  Is  proved. 

2 

Since  C*CR  , but  Is  defined  by  three  Inequalities,  the  fact 
that  It  can  be  generated  by  Intersecting  sets  foxmd  by  pairwise  coedalna- 
tlons  of  the  constraints  Is  Intuitively  satisfying. 

0.4  Olscusslon 

In  this  section  we  evaluate  the  bounds  for  two  simple  eases,  the 
degraded  dianneX  and  Blackwell's  exsnple. 

For  a degraded  channel,  the  cascade  (physically  degraded)  fennel 


▲ 
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is  in  p . since  in  general  I (X;yz)  > I (X;Y)  with  equality  for  cascade 

<1  — <1  j 

channels,  then  the  cascade  channel  achieves  the  infimua  in  (D.4)  and  ' 

Jq{X;Y2)  - (D.30)  | 

•( 

C*  then  becoBMS  yl 


r 


Rj  < Ig(X;Z)  (D.31) 

Rj^  + < i^(x,y)} 

2 

(D.31)  describes  a four-sided  region  in  R , vrtiich  we  have  sketched  in  Figure 
D.2  for  an  AMGN  channel  with  P > 10,  ■*  1 and  ~ 2.  Although  much  of 

the  reasoning  of  the  previous  sections  may  not  apply  to  arbitrary  continuous 
diannels,  the  results  are  all  easy  to  show  for  the  AWCN  case.  In  particular 
all  of  the  previous  theorems  are  easily  shown  for  degraded  channels  with- 
out using  any  of  the  convexity  theory,  idii^  required  among  other  things 
caag>actness  for  the  space  of  input  probabilities  and  channels, 
whl^  does  not  hold  in  the  continuous  case. 

The  region  C^(p)  described  by  equation  (D.13)  becoams  for  the 
cascnfta  channel 

C,  (p)  - U {<!»,»»,)«  R,  < l(Xi»|U) 

q«(l’  (D.32) 

R2<1(0|Z)  } 

Which  is  the  true  oapaolty  region.  This  region  is  aleo  sketdied  in  Figure 


=1.2iits 

.90iits 


Sato's  Bound  aid  Capacity  Beglon  of  AMGN  BC  with  P ■ 10,  9?  * 1 
and  al  ■ 2. 
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D.2.  Clearly,  in  this  case  the  bound  of  Theorem  D.2  is  strictly  tighter 
than  that  of  Theorem  0.1,  and  indeed  is  exact. 

Blackwell's  exan^le  is  a channel  described  by  a ternary  input 
alphabet  X « {0,1,2},  two  binary  output  alphabets  / - Z - {o,l},  and 
the  joint  cluuinel  probability  function  given  by 

p(0,ll0)  - p(l,0|l)  - p(l,ll2)  - 1 (D.33) 

This  channel  is  noiseless  and  not  degraded.  An  obvious  consequence  of 
the  noiselessness  is  that  the  set  P has  only  one  meinber,  the  actual 
channel.  Sato's  bound  for  this  channel  is  shown  in  Figure  D.3.  At  the 
time  of  the  appearance  of  this  result  in  [1] , it  was  the  smallest  known 
outer  bound  for  the  channel,  and  it  has  since  been  shown  (Gel'fand  [27]) 
that  the  whole  region  is  achievable  . Thus  while  Sato's  bound  is  loose 
for  degraded  channels,  it  has  been  shown  to  be  exact  for  at  least  one 
non-txivial  exas^le. 

Obviously  the  bound  of  Theorem  D.2  must  also  be  exact  for  this 

example. 

NOTEt  As  this  manuscript  ms  going  into  final  preparation,  a paper  ap- 
peared  in  which  a new  outer  bound  on  the  capacity  region  of  discrete 
amwryless  BC's  was  given  (Narton  (341).  we  have  not  had  time  to 
compare  the  bound  of  [34]  and  Theorem  D.2. 


j 

I 


-143- 


references 


1.  Van  der  Meulen,  E.C.  "A  Survey  of  Multi-way  Channels  in  Information 

Theory:  1961-1976,"  IEEE  Trans,  on  Inf.  Th.,  Vol.  IT-23,  pp.  1-37, 
Jan.  1977. 

2.  Schalkwijk,  J.P.M.  and  Kailath,  T.  "A  Coding  Scheme  for  Additive  Noise 

Channels  with  Feedback  - Part  i;  No  Bandwidth  Constraint,”  nam 
Trans,  on  Inf.  Th..  Vol.  lT-12,  pp.  172-182,  Apr.  1966. 

3.  Schalkwijk,  J.P.M.  "A  Coding  Scheaie  for  Additive  Noise  Channels  with 

Feedback  - Part  ll:  Band-Limited  Signals,"  IEEE  Trans,  on  Inf. 

Th. r Vol.  IT-12,  pp.  183-189,  Apr.  1966. 

4.  Cover,  T.M.  "Broadcast  Channels,"  IEEE  Trans,  on  Inf.  Th. . Vol.  IT-18, 

pp.  2-14,  Jan.  1972. 

5.  Bergmans,  P.P.  and  Cover,  T.M.  "Cooperative  Broadcasting,”  tbpp.  Triing. 

on  Inf.  Th..  Vol.  lT-20,  pp.  317-324,  May  1974. 

6.  Shannon,  C.E.  "Two-way  Cosnunicatlon  Channels,"  Proc.  4^  Berkeley  Svmp. 

Math.  Stat.  and  Prob..  Vol.  1,  pp.  611-644,  1961. 

7.  Bergmans,  P.P.  "A  Slagle  Converse  for  Broadcast  Channels  with  Additive 

Mhite  Gaussian  Noise,”  IEEE  Trans.  Inf.  Th..  Vol.  IT-19,  pp.  772 
-777,  Nov.  1973. 

8.  El-Gamal,  A.  "The  Feedback  Capacity  of  Degraded  Broadcast  Channels," 

IEEE  Trans,  cn  Inf.  Th. . Vol.  IT-24,  pp.  379-381,  Nay  1978. 

9.  Ahlswede,  R.  "On  Two-way  Cosmnu^catlon  Channels  and  a Problem  by 

Zaranklewlcs,"  Trans.  6^*  Prague  Conf.  on  inf.  Th..  stat,  Dec. 
Functicns.  Random  Processes,  pp.  23-37,  1971. 

10.  Liao,  H.  Multiple  Access  Channels,  Hi.D.  dissertation.  Dept,  of  B.E. , 

OBiv.  Hawaii,  1972. 

11.  Slepian,  D.  and  Nolf,  J.K.  "A  Coding  Theorem  for  Multiple  Access 

Channels  with  Correlated  Scarees,”  Bell  System  Tedinical  Jonm^^. 
Vol.  52,  pp.  1037-1076,  Sep.  1973. 


12.  Gaarder,  N.T.  and  Wolf,  J.K.  "The  Capacity  Region  of  a Multiple  Access 

Discrete  Maatoryless  Channel  can  Increase  with  Feedback,"  IKEl 
Trans,  on  Inf.  Th.,  Vol.  lT-21,  pp.  100-102,  Jan.  1975. 

13.  Cover,  T.M.  and  Leung-Tan-Checng,  S.X.  "A  Rate  Raglan  for  the  Multiple 


Access  Channel  with  Feedback,"  Stanford  Stat.  Tech.  Report  617, 
Sept.  1976. 


-144- 


14.  El-Gamal,  A.  "The  Capacity  of  a CIms  of  Broadcast  Channels,”  IEEE  Trans. 

on  Inf.  Th. , Vol.  IT-25,  pp.  166-169,  Mar.  1979. 

15.  Shannon,  C.E.  "A  Mathematical  Theory  of  Cossminication,"  Bell  System 

Technical  Journal.  Vol.  27,  pp.  379-423,623-656,  July  and  Oct.  1948 

16.  El-GasAl,  A.  "The  Capacity  of  the  Gaussian  Broadcast  Channel  with 

Feedback,"  Stanford  Dept.  Stat.  Tech.  Report  #25,  Jun.  1977. 

17.  Cover,  T.  "Open  Problems  in  Information  Theory,"  Proc.  1975  lEEE-OSSR 

Joint  workshop  on  Inf.  Th. , pp.  35-37,  1976. 

18.  Kozner,  J.  and  Marten,  K.  "The  Caap<urlson  of  TWo  Noisy  Channels,"  Trans. 

Coll,  on  Inf.  Th.,  Keszthely,  Hungary  1975,  pp.  411-423. 

19.  Cover,  T,  "Sene  Advances  in  Broadcast  Channels,"  Advances  in  Coantunication 

Systems,  Vol.  4,  A.  Viterbi,  ed. , San  Francisco;  Academic  Press,  1975. 

20.  Schalkwijk,  J.P.M.  "A  Class  of  Single  and  Optimal  Strategies  for  Block 

Coding  the  Binary  Symaetrlc  Channel  with  Noiseless  Feedback,"  ‘ 

IEEE  Trans,  on  inf.  Th.,  Vol.  lT-17,  pp.  283-287,  May  1971. 

21.  Ahlswede,  R.  "A  Constructive  Proof  of  the  Coding  T^gorem  for  Discrete 

Memory  less  Channels  with  Feedback,"  Trans.  6^  Prague  Ccof.  on 
mf.  Th. . Stat.  Decision  Functions.  Random  Processes,  pp. 39,50, 

1971. 

22.  Kashyap,  R.L.  "Feedback  Coding  Schemes  for  an  Additive  Noise  Channel 

wi^  a Noisy  Feedback  Link,”  IEEE  Trans.  Inf.  Th..Vel.  IT-14, 
pp.  471-480,  May  1968. 

I 

23.  Butsian,  S.  "A  general  Formulation  of  Linear  Feedback  Systems  with  i 

Solutions."  IEEE  Trans.  Inf.  Th. . Vol.  IT-15,  pp. 392-400,  ] 

May  1969.  | 

! 

24.  Gallager,  R.G.  Information  Theory  and  Reliable  Conmunication . New  York: 

Nlley,  1968. 

25.  Nyner,  A.D.  "Racwit  Results  in  the  Shannon  Theory,"  IEEE  Trans.  Inf. 

Th.,  Vol.  IT-20,  pp.  2-10,  Jan.  1974.  | 

26.  Rockafellar,  R.T.  Convex  Analysis,  Princeton;  Princeton  U.  1970.  l 

27.  Gel'fand,  S.I.  "Opacity  of  one  Broadcast  Channel,”  Problemy  Peredachi 

Informatsii . Vol.  13,  pp.  106-108,  Jul.-Sep.  1977.  i 


28.  Bergmans,  P.P.  "Random  Coding  Theorems  for  Broadcast  Channels  with  i 

Degraded  Components,”  IEEE  Trans,  mf.  Th.,  Vol.  IT-19,  pp. 

197-207,  Mar.  1973.  / 


'-145- 


29.  Berlekampr  E.R.  "Block  coding  for  the  Binary  Symmetric  Cheuinel  with 

Noiseless,  Delayless,  Feedback,"  in  Error  Correcting  Codes,  pp. 
61-85,  New  York:  Wiley,  1968. 

30.  Gallager,  R.G.  "Capacity  and  Coding  for  Degraded  Broadcast  Channels," 

Problemy  Peredachi  Informatsii,  Vol.  10,  pp.  3-14,  Jul.Sep.  1974. 

31.  Sato,  H.  "An  Outer  Bound  to  the  Capacity  Region  of  Broadcast 

Channels,  " IEEE  Tr^ul8 . on  Inf . Th . , Vol.  rr-24,  pp.  374-377, 

May  1978. 

32.  Shannon,  C.E.  "The  Zero  Error  Capacity  of  a Noisy  Channel,"  IRE  Trans. 

on  Inf.  Th.,  Vol.  IT-2,  pp.  8-19,  Sept.  1956. 

33.  Korner,  J.  2u\d  Martbn,  K.  "General  Broadcast  Channels  with  Degraded 
Message  Sets,"  IEEE  Trams,  on  Inf.  Th.,  Vol.  IT-23,  pp.  60-64, 
Jan.  1977. 

Maurton,  K.  "A  Coding  Theorem  for  the  Discrete  Memoryless  Broadcast 
Chamnel,"  IEEE  Trans,  on  Inf.  Th.,  Vol  IT-25,  pp.  306-311, 

May  1979. 


34. 


-146- 


BIOGW>PHlCaL  NOTE 


Lawrence  Ozarow  was  bom  in  the  Bronx,  N.Y.  on  October  31,  1949. 

He  received  the  B.S.  degree  in  electrical  engineering  fr<xn  Columbia 
Uhiversity  in  June  1970.  Prom  September  1970  to  June  1973  he  was 
enrolled  at  where  he  received  the  S.M.  and  E.E.  degrees  in 

electrical  engineering  in  June,  1973.  His  master's  research  was  in 
the  area  of  optical  ccsnninication . 

Proa  March  1973  to  September  1976  he  was  es^loyed  at  CNR,  Inc.  in 
Needham,  Ma. , working  in  the  area  of  design  and  analysis  of  coding  and 
modulation  techniques  for  communication  over  fading  channels. 

In  SeptMBber  1976  he  returned  to  M.I.T.  to  resume  his  graduate 
studies.  Prca  September  1976  to  June  1978  he  wets  a Resecurch  Assistant 
at  the  M.I.T.  Lincoln  Laboratories,  Lexington,  Ha.,  working  in  the 
area  of  satellite  comBunication.  From  June  1978  t June  1979  we  was 
a recipient  of  a Vinton  Hayes  Fellowship. 

Mr.  Osarow's  publications  to  date  are  "Packet  Switching  in  a Pro- 
cessing Satsllite,"  Proceedings  of  the  IEEE,  Jan.  1978  (with  J.K.  DeRosa), 
"Efficient  Packet  Satellite  Camunioatlon , " accepted  for  publication, 

IEEE  Transacations  on  Couwiunication  Theory  (with  J.K.  De  Rosa  and 
L.  Weiner),  "C^acity  Limits  for  Binary  Codes  in  the  Presence  of  Inter- 
ference," IEEE  Transactions  on  CoBmunication  Theory,  Feb.  1979  (with 
D. Chase),  and  "Combined  Voice  and  Packet  Communication  Via  Satellite," 
invited  paper,  I.C.C.  '79,  June,  1979  (with  J.K.  De  Rosa). 

Mr.  Ozarow  is  a member  of  Tau  Beta  Pi  and  Eta  Kappa  Nu. 


